Network is under initialization...
Network successfully initialized.
INFO: Downloading File to /root/PM-DARTS2/...

Succeed: Total num: 58, size: 521,803,881. OK num: 58(download 58 objects).

average speed 418112000(byte/s)

1.251472(s) elapsed
INFO: Downloading succeed.
WARN: ./requirements.txt not found, skip installing requirements.
Training with a single process on 1 GPUs.
Using native Torch AMP. Training in mixed precision.
Data processing configuration for current model + dataset:
	input_size: (3, 32, 32)
	interpolation: bilinear
	mean: (0.49139968, 0.48215827, 0.44653124)
	std: (0.24703233, 0.24348505, 0.26158768)
	crop_pct: 1.0
	crop_mode: center

-------------------------------
Learnable parameters
Student: 0.62M
Extra: 0.00M
-------------------------------
Scheduled epochs: 50
p_max: 0.2
search_space = s4
Using downloaded and verified file: /mnt/PM-DARTS2/data/cifar-10-python.tar.gz
Extracting /mnt/PM-DARTS2/data/cifar-10-python.tar.gz to /mnt/PM-DARTS2/data
Train: 0 [   0/390]  Loss: 2.324 (2.32)  Acc@1: 10.9375 (10.9375)  Acc@5: 46.8750 (46.8750)LR: 2.500e-02
Train: 0 [  50/390]  Loss: 2.304 (2.31)  Acc@1: 12.5000 (10.2635)  Acc@5: 54.6875 (50.1838)LR: 2.500e-02
Train: 0 [ 100/390]  Loss: 2.300 (2.30)  Acc@1:  9.3750 (10.5043)  Acc@5: 51.5625 (50.9437)LR: 2.500e-02
Train: 0 [ 150/390]  Loss: 2.052 (2.27)  Acc@1: 25.0000 (12.4897)  Acc@5: 71.8750 (56.0017)LR: 2.500e-02
Train: 0 [ 200/390]  Loss: 1.981 (2.20)  Acc@1: 17.1875 (14.5445)  Acc@5: 73.4375 (61.6294)LR: 2.500e-02
Train: 0 [ 250/390]  Loss: 1.884 (2.14)  Acc@1: 26.5625 (16.9945)  Acc@5: 79.6875 (65.8989)LR: 2.500e-02
Train: 0 [ 300/390]  Loss: 1.792 (2.09)  Acc@1: 25.0000 (19.1393)  Acc@5: 85.9375 (68.9992)LR: 2.500e-02
Train: 0 [ 350/390]  Loss: 1.771 (2.05)  Acc@1: 25.0000 (20.8289)  Acc@5: 82.8125 (71.4966)LR: 2.500e-02
Train: 0 [ 390/390]  Loss: 1.958 (2.02)  Acc@1: 22.5000 (21.9600)  Acc@5: 85.0000 (72.9720)LR: 2.500e-02
train_acc 21.960000
Valid: 0 [   0/390]  Loss: 1.732 (1.73)  Acc@1: 39.0625 (39.0625)  Acc@5: 90.6250 (90.6250)
Valid: 0 [  50/390]  Loss: 1.920 (1.81)  Acc@1: 29.6875 (33.1189)  Acc@5: 84.3750 (85.9375)
Valid: 0 [ 100/390]  Loss: 1.969 (1.81)  Acc@1: 23.4375 (32.7506)  Acc@5: 81.2500 (85.7983)
Valid: 0 [ 150/390]  Loss: 1.895 (1.81)  Acc@1: 34.3750 (32.8849)  Acc@5: 82.8125 (85.6064)
Valid: 0 [ 200/390]  Loss: 1.839 (1.81)  Acc@1: 32.8125 (32.5482)  Acc@5: 85.9375 (85.4167)
Valid: 0 [ 250/390]  Loss: 1.716 (1.81)  Acc@1: 34.3750 (32.5635)  Acc@5: 87.5000 (85.4893)
Valid: 0 [ 300/390]  Loss: 1.731 (1.80)  Acc@1: 35.9375 (32.5426)  Acc@5: 81.2500 (85.5741)
Valid: 0 [ 350/390]  Loss: 1.666 (1.80)  Acc@1: 32.8125 (32.6166)  Acc@5: 90.6250 (85.5012)
Valid: 0 [ 390/390]  Loss: 1.554 (1.80)  Acc@1: 42.5000 (32.6200)  Acc@5: 90.0000 (85.6120)
valid_acc 32.620000
epoch = 0   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4766, 0.5234],
        [0.4817, 0.5183],
        [0.4824, 0.5176],
        [0.4848, 0.5152],
        [0.4885, 0.5115],
        [0.4823, 0.5177],
        [0.4891, 0.5109],
        [0.4946, 0.5054],
        [0.4981, 0.5019],
        [0.4835, 0.5165],
        [0.4943, 0.5057],
        [0.4939, 0.5061],
        [0.4922, 0.5078],
        [0.4994, 0.5006]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4907, 0.5093],
        [0.4966, 0.5034],
        [0.4841, 0.5159],
        [0.5010, 0.4990],
        [0.4920, 0.5080],
        [0.4808, 0.5192],
        [0.4932, 0.5068],
        [0.4890, 0.5110],
        [0.4971, 0.5029],
        [0.4804, 0.5196],
        [0.4905, 0.5095],
        [0.4923, 0.5077],
        [0.4870, 0.5130],
        [0.4960, 0.5040]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 1 [   0/390]  Loss: 1.774 (1.77)  Acc@1: 40.6250 (40.6250)  Acc@5: 85.9375 (85.9375)LR: 2.498e-02
Train: 1 [  50/390]  Loss: 1.731 (1.74)  Acc@1: 39.0625 (33.4252)  Acc@5: 90.6250 (86.9792)LR: 2.498e-02
Train: 1 [ 100/390]  Loss: 1.393 (1.71)  Acc@1: 50.0000 (34.4833)  Acc@5: 90.6250 (87.4226)LR: 2.498e-02
Train: 1 [ 150/390]  Loss: 1.724 (1.69)  Acc@1: 28.1250 (34.8924)  Acc@5: 81.2500 (88.2554)LR: 2.498e-02
Train: 1 [ 200/390]  Loss: 1.648 (1.67)  Acc@1: 28.1250 (35.5488)  Acc@5: 90.6250 (88.5805)LR: 2.498e-02
Train: 1 [ 250/390]  Loss: 1.502 (1.65)  Acc@1: 37.5000 (36.6160)  Acc@5: 98.4375 (89.0812)LR: 2.498e-02
Train: 1 [ 300/390]  Loss: 1.466 (1.63)  Acc@1: 40.6250 (37.4740)  Acc@5: 93.7500 (89.3843)LR: 2.498e-02
Train: 1 [ 350/390]  Loss: 1.481 (1.62)  Acc@1: 45.3125 (38.2657)  Acc@5: 93.7500 (89.5967)LR: 2.498e-02
Train: 1 [ 390/390]  Loss: 1.638 (1.60)  Acc@1: 45.0000 (38.9000)  Acc@5: 90.0000 (89.6800)LR: 2.498e-02
train_acc 38.900000
Valid: 1 [   0/390]  Loss: 1.646 (1.65)  Acc@1: 40.6250 (40.6250)  Acc@5: 87.5000 (87.5000)
Valid: 1 [  50/390]  Loss: 1.795 (1.55)  Acc@1: 40.6250 (42.0650)  Acc@5: 89.0625 (91.1152)
Valid: 1 [ 100/390]  Loss: 1.562 (1.57)  Acc@1: 50.0000 (41.8781)  Acc@5: 95.3125 (90.8261)
Valid: 1 [ 150/390]  Loss: 1.866 (1.57)  Acc@1: 40.6250 (42.1151)  Acc@5: 87.5000 (90.7595)
Valid: 1 [ 200/390]  Loss: 1.491 (1.56)  Acc@1: 29.6875 (42.2575)  Acc@5: 93.7500 (90.8116)
Valid: 1 [ 250/390]  Loss: 1.661 (1.57)  Acc@1: 34.3750 (42.1813)  Acc@5: 89.0625 (90.7744)
Valid: 1 [ 300/390]  Loss: 1.615 (1.56)  Acc@1: 35.9375 (42.1096)  Acc@5: 87.5000 (90.7496)
Valid: 1 [ 350/390]  Loss: 1.828 (1.57)  Acc@1: 34.3750 (41.9382)  Acc@5: 87.5000 (90.7229)
Valid: 1 [ 390/390]  Loss: 1.712 (1.56)  Acc@1: 47.5000 (42.0760)  Acc@5: 92.5000 (90.8040)
valid_acc 42.076000
epoch = 1   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4567, 0.5433],
        [0.4612, 0.5388],
        [0.4669, 0.5331],
        [0.4644, 0.5356],
        [0.4803, 0.5197],
        [0.4632, 0.5368],
        [0.4702, 0.5298],
        [0.4829, 0.5171],
        [0.4898, 0.5102],
        [0.4652, 0.5348],
        [0.4775, 0.5225],
        [0.4856, 0.5144],
        [0.4800, 0.5200],
        [0.4865, 0.5135]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4727, 0.5273],
        [0.4847, 0.5153],
        [0.4680, 0.5320],
        [0.4916, 0.5084],
        [0.4823, 0.5177],
        [0.4725, 0.5275],
        [0.4799, 0.5201],
        [0.4790, 0.5210],
        [0.4943, 0.5057],
        [0.4613, 0.5387],
        [0.4825, 0.5175],
        [0.4916, 0.5084],
        [0.4825, 0.5175],
        [0.4925, 0.5075]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 2 [   0/390]  Loss: 1.488 (1.49)  Acc@1: 50.0000 (50.0000)  Acc@5: 92.1875 (92.1875)LR: 2.491e-02
Train: 2 [  50/390]  Loss: 1.464 (1.47)  Acc@1: 45.3125 (45.0980)  Acc@5: 87.5000 (91.8811)LR: 2.491e-02
Train: 2 [ 100/390]  Loss: 1.318 (1.43)  Acc@1: 48.4375 (46.4418)  Acc@5: 96.8750 (92.6980)LR: 2.491e-02
Train: 2 [ 150/390]  Loss: 1.551 (1.42)  Acc@1: 46.8750 (47.2372)  Acc@5: 87.5000 (92.8291)LR: 2.491e-02
Train: 2 [ 200/390]  Loss: 1.294 (1.42)  Acc@1: 51.5625 (47.6135)  Acc@5: 95.3125 (92.7938)LR: 2.491e-02
Train: 2 [ 250/390]  Loss: 1.505 (1.41)  Acc@1: 43.7500 (48.0017)  Acc@5: 92.1875 (92.8225)LR: 2.491e-02
Train: 2 [ 300/390]  Loss: 1.428 (1.40)  Acc@1: 51.5625 (48.1312)  Acc@5: 90.6250 (92.7949)LR: 2.491e-02
Train: 2 [ 350/390]  Loss: 1.460 (1.40)  Acc@1: 51.5625 (48.3885)  Acc@5: 90.6250 (92.7796)LR: 2.491e-02
Train: 2 [ 390/390]  Loss: 1.276 (1.39)  Acc@1: 57.5000 (48.8640)  Acc@5: 92.5000 (92.9280)LR: 2.491e-02
train_acc 48.864000
Valid: 2 [   0/390]  Loss: 1.541 (1.54)  Acc@1: 40.6250 (40.6250)  Acc@5: 89.0625 (89.0625)
Valid: 2 [  50/390]  Loss: 1.418 (1.37)  Acc@1: 56.2500 (50.0000)  Acc@5: 85.9375 (92.8615)
Valid: 2 [ 100/390]  Loss: 1.473 (1.38)  Acc@1: 35.9375 (49.7370)  Acc@5: 96.8750 (93.1467)
Valid: 2 [ 150/390]  Loss: 1.469 (1.39)  Acc@1: 50.0000 (49.5447)  Acc@5: 92.1875 (92.8187)
Valid: 2 [ 200/390]  Loss: 1.100 (1.39)  Acc@1: 56.2500 (49.3470)  Acc@5: 95.3125 (92.8016)
Valid: 2 [ 250/390]  Loss: 1.610 (1.39)  Acc@1: 43.7500 (49.3028)  Acc@5: 95.3125 (92.7913)
Valid: 2 [ 300/390]  Loss: 1.479 (1.39)  Acc@1: 48.4375 (49.3355)  Acc@5: 90.6250 (92.7897)
Valid: 2 [ 350/390]  Loss: 1.340 (1.39)  Acc@1: 56.2500 (49.2566)  Acc@5: 92.1875 (92.9265)
Valid: 2 [ 390/390]  Loss: 1.485 (1.39)  Acc@1: 47.5000 (49.2720)  Acc@5: 90.0000 (92.9520)
valid_acc 49.272000
epoch = 2   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4394, 0.5606],
        [0.4408, 0.5592],
        [0.4521, 0.5479],
        [0.4385, 0.5615],
        [0.4733, 0.5267],
        [0.4527, 0.5473],
        [0.4455, 0.5545],
        [0.4686, 0.5314],
        [0.4799, 0.5201],
        [0.4539, 0.5461],
        [0.4648, 0.5352],
        [0.4811, 0.5189],
        [0.4691, 0.5309],
        [0.4788, 0.5212]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4536, 0.5464],
        [0.4678, 0.5322],
        [0.4534, 0.5466],
        [0.4837, 0.5163],
        [0.4702, 0.5298],
        [0.4616, 0.5384],
        [0.4689, 0.5311],
        [0.4693, 0.5307],
        [0.4825, 0.5175],
        [0.4508, 0.5492],
        [0.4788, 0.5212],
        [0.4882, 0.5118],
        [0.4725, 0.5275],
        [0.4875, 0.5125]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 3 [   0/390]  Loss: 1.209 (1.21)  Acc@1: 54.6875 (54.6875)  Acc@5: 96.8750 (96.8750)LR: 2.479e-02
Train: 3 [  50/390]  Loss: 1.355 (1.26)  Acc@1: 48.4375 (53.9828)  Acc@5: 95.3125 (94.9142)LR: 2.479e-02
Train: 3 [ 100/390]  Loss: 1.345 (1.28)  Acc@1: 59.3750 (53.7283)  Acc@5: 93.7500 (94.4616)LR: 2.479e-02
Train: 3 [ 150/390]  Loss: 1.449 (1.26)  Acc@1: 54.6875 (54.7599)  Acc@5: 92.1875 (94.4847)LR: 2.479e-02
Train: 3 [ 200/390]  Loss: 1.208 (1.25)  Acc@1: 59.3750 (54.8896)  Acc@5: 96.8750 (94.6284)LR: 2.479e-02
Train: 3 [ 250/390]  Loss: 1.217 (1.24)  Acc@1: 53.1250 (55.2540)  Acc@5: 93.7500 (94.6838)LR: 2.479e-02
Train: 3 [ 300/390]  Loss: 1.120 (1.23)  Acc@1: 57.8125 (55.5388)  Acc@5: 95.3125 (94.6740)LR: 2.479e-02
Train: 3 [ 350/390]  Loss: 0.9194 (1.22)  Acc@1: 68.7500 (55.8672)  Acc@5: 100.0000 (94.7249)LR: 2.479e-02
Train: 3 [ 390/390]  Loss: 1.291 (1.22)  Acc@1: 50.0000 (56.0080)  Acc@5: 97.5000 (94.7520)LR: 2.479e-02
train_acc 56.008000
Valid: 3 [   0/390]  Loss: 1.189 (1.19)  Acc@1: 56.2500 (56.2500)  Acc@5: 92.1875 (92.1875)
Valid: 3 [  50/390]  Loss: 1.293 (1.18)  Acc@1: 57.8125 (57.1691)  Acc@5: 93.7500 (95.5576)
Valid: 3 [ 100/390]  Loss: 0.9731 (1.16)  Acc@1: 67.1875 (58.0910)  Acc@5: 96.8750 (95.7457)
Valid: 3 [ 150/390]  Loss: 1.183 (1.15)  Acc@1: 56.2500 (58.2885)  Acc@5: 93.7500 (95.5608)
Valid: 3 [ 200/390]  Loss: 1.368 (1.15)  Acc@1: 54.6875 (58.2245)  Acc@5: 92.1875 (95.5302)
Valid: 3 [ 250/390]  Loss: 1.360 (1.15)  Acc@1: 53.1250 (58.2856)  Acc@5: 93.7500 (95.5864)
Valid: 3 [ 300/390]  Loss: 1.130 (1.16)  Acc@1: 57.8125 (58.1188)  Acc@5: 95.3125 (95.5980)
Valid: 3 [ 350/390]  Loss: 1.201 (1.16)  Acc@1: 56.2500 (58.1642)  Acc@5: 89.0625 (95.5128)
Valid: 3 [ 390/390]  Loss: 1.071 (1.16)  Acc@1: 65.0000 (58.1280)  Acc@5: 97.5000 (95.4040)
valid_acc 58.128000
epoch = 3   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4237, 0.5763],
        [0.4252, 0.5748],
        [0.4358, 0.5642],
        [0.4189, 0.5811],
        [0.4686, 0.5314],
        [0.4345, 0.5655],
        [0.4256, 0.5744],
        [0.4602, 0.5398],
        [0.4729, 0.5271],
        [0.4407, 0.5593],
        [0.4573, 0.5427],
        [0.4719, 0.5281],
        [0.4603, 0.5397],
        [0.4763, 0.5237]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4353, 0.5647],
        [0.4524, 0.5476],
        [0.4465, 0.5535],
        [0.4683, 0.5317],
        [0.4658, 0.5342],
        [0.4478, 0.5522],
        [0.4563, 0.5437],
        [0.4630, 0.5370],
        [0.4758, 0.5242],
        [0.4346, 0.5654],
        [0.4731, 0.5269],
        [0.4801, 0.5199],
        [0.4630, 0.5370],
        [0.4825, 0.5175]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 4 [   0/390]  Loss: 1.346 (1.35)  Acc@1: 51.5625 (51.5625)  Acc@5: 93.7500 (93.7500)LR: 2.462e-02
Train: 4 [  50/390]  Loss: 1.003 (1.14)  Acc@1: 70.3125 (58.6397)  Acc@5: 93.7500 (95.7414)LR: 2.462e-02
Train: 4 [ 100/390]  Loss: 1.363 (1.15)  Acc@1: 53.1250 (58.5087)  Acc@5: 95.3125 (95.9004)LR: 2.462e-02
Train: 4 [ 150/390]  Loss: 1.060 (1.14)  Acc@1: 65.6250 (59.1577)  Acc@5: 98.4375 (95.8506)LR: 2.462e-02
Train: 4 [ 200/390]  Loss: 1.140 (1.13)  Acc@1: 46.8750 (59.2895)  Acc@5: 95.3125 (95.7634)LR: 2.462e-02
Train: 4 [ 250/390]  Loss: 1.094 (1.12)  Acc@1: 53.1250 (59.8232)  Acc@5: 95.3125 (95.7358)LR: 2.462e-02
Train: 4 [ 300/390]  Loss: 1.012 (1.11)  Acc@1: 60.9375 (60.2108)  Acc@5: 95.3125 (95.7797)LR: 2.462e-02
Train: 4 [ 350/390]  Loss: 1.092 (1.11)  Acc@1: 53.1250 (60.1852)  Acc@5: 96.8750 (95.8645)LR: 2.462e-02
Train: 4 [ 390/390]  Loss: 0.9443 (1.10)  Acc@1: 67.5000 (60.4960)  Acc@5: 97.5000 (95.8840)LR: 2.462e-02
train_acc 60.496000
Valid: 4 [   0/390]  Loss: 0.9023 (0.902)  Acc@1: 68.7500 (68.7500)  Acc@5: 96.8750 (96.8750)
Valid: 4 [  50/390]  Loss: 1.049 (1.07)  Acc@1: 57.8125 (61.9792)  Acc@5: 100.0000 (95.8946)
Valid: 4 [ 100/390]  Loss: 0.9736 (1.06)  Acc@1: 60.9375 (61.8038)  Acc@5: 98.4375 (96.2252)
Valid: 4 [ 150/390]  Loss: 1.126 (1.06)  Acc@1: 57.8125 (61.9102)  Acc@5: 96.8750 (96.1093)
Valid: 4 [ 200/390]  Loss: 0.9854 (1.06)  Acc@1: 67.1875 (61.9325)  Acc@5: 95.3125 (96.0743)
Valid: 4 [ 250/390]  Loss: 1.086 (1.05)  Acc@1: 60.9375 (62.0767)  Acc@5: 92.1875 (96.1716)
Valid: 4 [ 300/390]  Loss: 1.374 (1.06)  Acc@1: 48.4375 (61.8978)  Acc@5: 90.6250 (96.1742)
Valid: 4 [ 350/390]  Loss: 1.037 (1.06)  Acc@1: 65.6250 (61.8634)  Acc@5: 96.8750 (96.1449)
Valid: 4 [ 390/390]  Loss: 1.185 (1.06)  Acc@1: 60.0000 (61.9120)  Acc@5: 90.0000 (96.1200)
valid_acc 61.912000
epoch = 4   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4091, 0.5909],
        [0.4073, 0.5927],
        [0.4151, 0.5849],
        [0.3976, 0.6024],
        [0.4575, 0.5425],
        [0.4207, 0.5793],
        [0.4071, 0.5929],
        [0.4507, 0.5493],
        [0.4662, 0.5338],
        [0.4338, 0.5662],
        [0.4483, 0.5517],
        [0.4636, 0.5364],
        [0.4435, 0.5565],
        [0.4679, 0.5321]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4155, 0.5845],
        [0.4382, 0.5618],
        [0.4333, 0.5667],
        [0.4601, 0.5399],
        [0.4604, 0.5396],
        [0.4368, 0.5632],
        [0.4452, 0.5548],
        [0.4494, 0.5506],
        [0.4661, 0.5339],
        [0.4217, 0.5783],
        [0.4635, 0.5365],
        [0.4651, 0.5349],
        [0.4554, 0.5446],
        [0.4710, 0.5290]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 5 [   0/390]  Loss: 0.9083 (0.908)  Acc@1: 70.3125 (70.3125)  Acc@5: 96.8750 (96.8750)LR: 2.441e-02
Train: 5 [  50/390]  Loss: 0.9788 (1.03)  Acc@1: 64.0625 (63.6949)  Acc@5: 96.8750 (96.2623)LR: 2.441e-02
Train: 5 [ 100/390]  Loss: 1.064 (1.03)  Acc@1: 71.8750 (64.0470)  Acc@5: 98.4375 (96.3800)LR: 2.441e-02
Train: 5 [ 150/390]  Loss: 0.7718 (1.02)  Acc@1: 82.8125 (64.0211)  Acc@5: 96.8750 (96.4507)LR: 2.441e-02
Train: 5 [ 200/390]  Loss: 1.044 (1.02)  Acc@1: 60.9375 (63.8837)  Acc@5: 96.8750 (96.4552)LR: 2.441e-02
Train: 5 [ 250/390]  Loss: 0.9061 (1.01)  Acc@1: 64.0625 (64.0500)  Acc@5: 96.8750 (96.5077)LR: 2.441e-02
Train: 5 [ 300/390]  Loss: 1.192 (1.01)  Acc@1: 54.6875 (64.1404)  Acc@5: 96.8750 (96.5687)LR: 2.441e-02
Train: 5 [ 350/390]  Loss: 1.112 (1.01)  Acc@1: 64.0625 (64.0002)  Acc@5: 92.1875 (96.5589)LR: 2.441e-02
Train: 5 [ 390/390]  Loss: 1.205 (1.01)  Acc@1: 62.5000 (64.1200)  Acc@5: 97.5000 (96.6040)LR: 2.441e-02
train_acc 64.120000
Valid: 5 [   0/390]  Loss: 1.034 (1.03)  Acc@1: 67.1875 (67.1875)  Acc@5: 95.3125 (95.3125)
Valid: 5 [  50/390]  Loss: 0.7305 (1.09)  Acc@1: 78.1250 (62.6838)  Acc@5: 96.8750 (95.2512)
Valid: 5 [ 100/390]  Loss: 1.115 (1.07)  Acc@1: 62.5000 (62.7785)  Acc@5: 93.7500 (95.3434)
Valid: 5 [ 150/390]  Loss: 1.129 (1.08)  Acc@1: 67.1875 (62.7173)  Acc@5: 93.7500 (95.2401)
Valid: 5 [ 200/390]  Loss: 1.052 (1.07)  Acc@1: 65.6250 (62.6943)  Acc@5: 98.4375 (95.4835)
Valid: 5 [ 250/390]  Loss: 0.9458 (1.07)  Acc@1: 71.8750 (62.7179)  Acc@5: 100.0000 (95.5864)
Valid: 5 [ 300/390]  Loss: 1.045 (1.07)  Acc@1: 67.1875 (62.8167)  Acc@5: 96.8750 (95.6966)
Valid: 5 [ 350/390]  Loss: 1.282 (1.07)  Acc@1: 60.9375 (62.8339)  Acc@5: 93.7500 (95.7220)
Valid: 5 [ 390/390]  Loss: 1.072 (1.07)  Acc@1: 65.0000 (63.0080)  Acc@5: 95.0000 (95.7520)
valid_acc 63.008000
epoch = 5   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3956, 0.6044],
        [0.3929, 0.6071],
        [0.3989, 0.6011],
        [0.3809, 0.6191],
        [0.4444, 0.5556],
        [0.4084, 0.5916],
        [0.3856, 0.6144],
        [0.4375, 0.5625],
        [0.4579, 0.5421],
        [0.4259, 0.5741],
        [0.4349, 0.5651],
        [0.4574, 0.5426],
        [0.4311, 0.5689],
        [0.4622, 0.5378]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3979, 0.6021],
        [0.4188, 0.5812],
        [0.4217, 0.5783],
        [0.4469, 0.5531],
        [0.4539, 0.5461],
        [0.4186, 0.5814],
        [0.4366, 0.5634],
        [0.4435, 0.5565],
        [0.4634, 0.5366],
        [0.4166, 0.5834],
        [0.4542, 0.5458],
        [0.4540, 0.5460],
        [0.4501, 0.5499],
        [0.4629, 0.5371]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 6 [   0/390]  Loss: 0.7591 (0.759)  Acc@1: 78.1250 (78.1250)  Acc@5: 100.0000 (100.0000)LR: 2.416e-02
Train: 6 [  50/390]  Loss: 0.7929 (0.938)  Acc@1: 82.8125 (67.3407)  Acc@5: 98.4375 (97.2733)LR: 2.416e-02
Train: 6 [ 100/390]  Loss: 0.7837 (0.947)  Acc@1: 73.4375 (66.5377)  Acc@5: 100.0000 (97.1535)LR: 2.416e-02
Train: 6 [ 150/390]  Loss: 0.9068 (0.954)  Acc@1: 67.1875 (66.3907)  Acc@5: 100.0000 (96.9681)LR: 2.416e-02
Train: 6 [ 200/390]  Loss: 0.9365 (0.949)  Acc@1: 64.0625 (66.5034)  Acc@5: 98.4375 (97.0771)LR: 2.416e-02
Train: 6 [ 250/390]  Loss: 1.113 (0.945)  Acc@1: 67.1875 (66.6210)  Acc@5: 93.7500 (97.1302)LR: 2.416e-02
Train: 6 [ 300/390]  Loss: 0.9879 (0.943)  Acc@1: 70.3125 (66.5750)  Acc@5: 93.7500 (97.1138)LR: 2.416e-02
Train: 6 [ 350/390]  Loss: 0.9069 (0.932)  Acc@1: 65.6250 (66.9916)  Acc@5: 96.8750 (97.2356)LR: 2.416e-02
Train: 6 [ 390/390]  Loss: 0.8880 (0.926)  Acc@1: 62.5000 (67.1640)  Acc@5: 100.0000 (97.2680)LR: 2.416e-02
train_acc 67.164000
Valid: 6 [   0/390]  Loss: 0.7466 (0.747)  Acc@1: 71.8750 (71.8750)  Acc@5: 100.0000 (100.0000)
Valid: 6 [  50/390]  Loss: 0.9136 (0.921)  Acc@1: 65.6250 (65.8088)  Acc@5: 96.8750 (97.5184)
Valid: 6 [ 100/390]  Loss: 0.8633 (0.926)  Acc@1: 67.1875 (66.4295)  Acc@5: 98.4375 (97.1689)
Valid: 6 [ 150/390]  Loss: 1.078 (0.932)  Acc@1: 64.0625 (66.4425)  Acc@5: 98.4375 (97.1958)
Valid: 6 [ 200/390]  Loss: 0.8794 (0.937)  Acc@1: 70.3125 (66.3091)  Acc@5: 96.8750 (97.2404)
Valid: 6 [ 250/390]  Loss: 1.083 (0.937)  Acc@1: 59.3750 (66.4280)  Acc@5: 95.3125 (97.2049)
Valid: 6 [ 300/390]  Loss: 1.038 (0.942)  Acc@1: 60.9375 (66.4088)  Acc@5: 96.8750 (97.1761)
Valid: 6 [ 350/390]  Loss: 0.7563 (0.939)  Acc@1: 71.8750 (66.4352)  Acc@5: 98.4375 (97.1777)
Valid: 6 [ 390/390]  Loss: 0.8506 (0.939)  Acc@1: 62.5000 (66.4000)  Acc@5: 97.5000 (97.1720)
valid_acc 66.400000
epoch = 6   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3817, 0.6183],
        [0.3775, 0.6225],
        [0.3798, 0.6202],
        [0.3586, 0.6414],
        [0.4328, 0.5672],
        [0.3962, 0.6038],
        [0.3675, 0.6325],
        [0.4346, 0.5654],
        [0.4473, 0.5527],
        [0.4143, 0.5857],
        [0.4248, 0.5752],
        [0.4470, 0.5530],
        [0.4281, 0.5719],
        [0.4565, 0.5435]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3827, 0.6173],
        [0.4042, 0.5958],
        [0.4094, 0.5906],
        [0.4353, 0.5647],
        [0.4478, 0.5522],
        [0.4018, 0.5982],
        [0.4296, 0.5704],
        [0.4412, 0.5588],
        [0.4618, 0.5382],
        [0.4008, 0.5992],
        [0.4464, 0.5536],
        [0.4436, 0.5564],
        [0.4444, 0.5556],
        [0.4607, 0.5393]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 7 [   0/390]  Loss: 0.8845 (0.885)  Acc@1: 73.4375 (73.4375)  Acc@5: 92.1875 (92.1875)LR: 2.386e-02
Train: 7 [  50/390]  Loss: 0.9847 (0.884)  Acc@1: 59.3750 (68.2904)  Acc@5: 98.4375 (97.6409)LR: 2.386e-02
Train: 7 [ 100/390]  Loss: 0.8514 (0.874)  Acc@1: 65.6250 (68.3478)  Acc@5: 96.8750 (97.7413)LR: 2.386e-02
Train: 7 [ 150/390]  Loss: 0.9716 (0.876)  Acc@1: 62.5000 (68.2637)  Acc@5: 98.4375 (97.5786)LR: 2.386e-02
Train: 7 [ 200/390]  Loss: 0.7230 (0.873)  Acc@1: 67.1875 (68.6412)  Acc@5: 100.0000 (97.5746)LR: 2.386e-02
Train: 7 [ 250/390]  Loss: 0.9339 (0.861)  Acc@1: 70.3125 (69.3601)  Acc@5: 95.3125 (97.6158)LR: 2.386e-02
Train: 7 [ 300/390]  Loss: 0.7520 (0.856)  Acc@1: 75.0000 (69.5754)  Acc@5: 98.4375 (97.6900)LR: 2.386e-02
Train: 7 [ 350/390]  Loss: 0.7659 (0.850)  Acc@1: 71.8750 (69.8095)  Acc@5: 98.4375 (97.6763)LR: 2.386e-02
Train: 7 [ 390/390]  Loss: 0.8544 (0.847)  Acc@1: 77.5000 (69.9240)  Acc@5: 95.0000 (97.6880)LR: 2.386e-02
train_acc 69.924000
Valid: 7 [   0/390]  Loss: 0.8340 (0.834)  Acc@1: 70.3125 (70.3125)  Acc@5: 93.7500 (93.7500)
Valid: 7 [  50/390]  Loss: 0.8867 (0.856)  Acc@1: 68.7500 (69.7917)  Acc@5: 98.4375 (97.7022)
Valid: 7 [ 100/390]  Loss: 0.7817 (0.861)  Acc@1: 73.4375 (69.3843)  Acc@5: 98.4375 (97.8342)
Valid: 7 [ 150/390]  Loss: 0.9697 (0.847)  Acc@1: 65.6250 (70.2504)  Acc@5: 96.8750 (97.8891)
Valid: 7 [ 200/390]  Loss: 0.8530 (0.855)  Acc@1: 67.1875 (70.1182)  Acc@5: 96.8750 (97.7456)
Valid: 7 [ 250/390]  Loss: 0.7524 (0.849)  Acc@1: 73.4375 (70.2191)  Acc@5: 98.4375 (97.7901)
Valid: 7 [ 300/390]  Loss: 0.9600 (0.845)  Acc@1: 65.6250 (70.3125)  Acc@5: 98.4375 (97.8509)
Valid: 7 [ 350/390]  Loss: 0.7754 (0.845)  Acc@1: 68.7500 (70.4060)  Acc@5: 98.4375 (97.8410)
Valid: 7 [ 390/390]  Loss: 0.5562 (0.845)  Acc@1: 82.5000 (70.4720)  Acc@5: 100.0000 (97.8000)
valid_acc 70.472000
epoch = 7   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3709, 0.6291],
        [0.3657, 0.6343],
        [0.3649, 0.6351],
        [0.3388, 0.6612],
        [0.4170, 0.5830],
        [0.3958, 0.6042],
        [0.3499, 0.6501],
        [0.4309, 0.5691],
        [0.4386, 0.5614],
        [0.4031, 0.5969],
        [0.4116, 0.5884],
        [0.4408, 0.5592],
        [0.4135, 0.5865],
        [0.4539, 0.5461]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3686, 0.6314],
        [0.3877, 0.6123],
        [0.4004, 0.5996],
        [0.4283, 0.5717],
        [0.4426, 0.5574],
        [0.3871, 0.6129],
        [0.4240, 0.5760],
        [0.4326, 0.5674],
        [0.4575, 0.5425],
        [0.3882, 0.6118],
        [0.4341, 0.5659],
        [0.4351, 0.5649],
        [0.4390, 0.5610],
        [0.4570, 0.5430]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 8 [   0/390]  Loss: 0.7239 (0.724)  Acc@1: 68.7500 (68.7500)  Acc@5: 100.0000 (100.0000)LR: 2.352e-02
Train: 8 [  50/390]  Loss: 0.8095 (0.782)  Acc@1: 78.1250 (72.9779)  Acc@5: 95.3125 (97.8860)LR: 2.352e-02
Train: 8 [ 100/390]  Loss: 0.8423 (0.780)  Acc@1: 76.5625 (72.9425)  Acc@5: 96.8750 (98.1590)LR: 2.352e-02
Train: 8 [ 150/390]  Loss: 0.6739 (0.790)  Acc@1: 78.1250 (72.4131)  Acc@5: 100.0000 (98.1271)LR: 2.352e-02
Train: 8 [ 200/390]  Loss: 0.8120 (0.789)  Acc@1: 64.0625 (72.3958)  Acc@5: 98.4375 (98.0877)LR: 2.352e-02
Train: 8 [ 250/390]  Loss: 0.9132 (0.793)  Acc@1: 67.1875 (72.1489)  Acc@5: 96.8750 (98.0453)LR: 2.352e-02
Train: 8 [ 300/390]  Loss: 0.9156 (0.793)  Acc@1: 75.0000 (72.2124)  Acc@5: 98.4375 (98.0430)LR: 2.352e-02
Train: 8 [ 350/390]  Loss: 0.6508 (0.790)  Acc@1: 78.1250 (72.3647)  Acc@5: 98.4375 (98.0369)LR: 2.352e-02
Train: 8 [ 390/390]  Loss: 0.7440 (0.786)  Acc@1: 75.0000 (72.4960)  Acc@5: 100.0000 (98.0480)LR: 2.352e-02
train_acc 72.496000
Valid: 8 [   0/390]  Loss: 0.6738 (0.674)  Acc@1: 78.1250 (78.1250)  Acc@5: 100.0000 (100.0000)
Valid: 8 [  50/390]  Loss: 0.9561 (0.786)  Acc@1: 68.7500 (72.3958)  Acc@5: 95.3125 (98.3762)
Valid: 8 [ 100/390]  Loss: 0.6899 (0.798)  Acc@1: 79.6875 (71.9833)  Acc@5: 100.0000 (98.0817)
Valid: 8 [ 150/390]  Loss: 0.9380 (0.792)  Acc@1: 68.7500 (72.0302)  Acc@5: 96.8750 (98.1892)
Valid: 8 [ 200/390]  Loss: 0.8779 (0.788)  Acc@1: 68.7500 (72.3181)  Acc@5: 96.8750 (98.1032)
Valid: 8 [ 250/390]  Loss: 0.7095 (0.793)  Acc@1: 73.4375 (72.2049)  Acc@5: 100.0000 (97.9893)
Valid: 8 [ 300/390]  Loss: 0.8750 (0.795)  Acc@1: 68.7500 (72.1294)  Acc@5: 96.8750 (97.9755)
Valid: 8 [ 350/390]  Loss: 0.5327 (0.792)  Acc@1: 82.8125 (72.2578)  Acc@5: 100.0000 (97.9834)
Valid: 8 [ 390/390]  Loss: 0.7803 (0.794)  Acc@1: 72.5000 (72.0840)  Acc@5: 100.0000 (97.9520)
valid_acc 72.084000
epoch = 8   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3615, 0.6385],
        [0.3530, 0.6470],
        [0.3507, 0.6493],
        [0.3202, 0.6798],
        [0.4062, 0.5938],
        [0.3908, 0.6092],
        [0.3313, 0.6687],
        [0.4267, 0.5733],
        [0.4292, 0.5708],
        [0.3923, 0.6077],
        [0.4011, 0.5989],
        [0.4334, 0.5666],
        [0.4101, 0.5899],
        [0.4484, 0.5516]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3529, 0.6471],
        [0.3766, 0.6234],
        [0.3926, 0.6074],
        [0.4191, 0.5809],
        [0.4360, 0.5640],
        [0.3754, 0.6246],
        [0.4102, 0.5898],
        [0.4224, 0.5776],
        [0.4489, 0.5511],
        [0.3836, 0.6164],
        [0.4178, 0.5822],
        [0.4196, 0.5804],
        [0.4325, 0.5675],
        [0.4506, 0.5494]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 9 [   0/390]  Loss: 0.5750 (0.575)  Acc@1: 76.5625 (76.5625)  Acc@5: 98.4375 (98.4375)LR: 2.313e-02
Train: 9 [  50/390]  Loss: 0.7015 (0.720)  Acc@1: 78.1250 (74.8775)  Acc@5: 100.0000 (98.3456)LR: 2.313e-02
Train: 9 [ 100/390]  Loss: 0.6126 (0.735)  Acc@1: 76.5625 (74.0099)  Acc@5: 100.0000 (98.1745)LR: 2.313e-02
Train: 9 [ 150/390]  Loss: 0.6065 (0.736)  Acc@1: 81.2500 (74.1101)  Acc@5: 96.8750 (98.2099)LR: 2.313e-02
Train: 9 [ 200/390]  Loss: 0.8294 (0.730)  Acc@1: 68.7500 (74.3548)  Acc@5: 98.4375 (98.2743)LR: 2.313e-02
Train: 9 [ 250/390]  Loss: 0.7282 (0.725)  Acc@1: 78.1250 (74.6389)  Acc@5: 98.4375 (98.3068)LR: 2.313e-02
Train: 9 [ 300/390]  Loss: 0.8678 (0.723)  Acc@1: 71.8750 (74.8754)  Acc@5: 96.8750 (98.2870)LR: 2.313e-02
Train: 9 [ 350/390]  Loss: 0.7986 (0.728)  Acc@1: 73.4375 (74.6483)  Acc@5: 98.4375 (98.2594)LR: 2.313e-02
Train: 9 [ 390/390]  Loss: 0.9210 (0.728)  Acc@1: 67.5000 (74.6520)  Acc@5: 92.5000 (98.2600)LR: 2.313e-02
train_acc 74.652000
Valid: 9 [   0/390]  Loss: 0.8950 (0.895)  Acc@1: 65.6250 (65.6250)  Acc@5: 96.8750 (96.8750)
Valid: 9 [  50/390]  Loss: 0.6250 (0.831)  Acc@1: 81.2500 (71.2010)  Acc@5: 100.0000 (97.9779)
Valid: 9 [ 100/390]  Loss: 0.9878 (0.829)  Acc@1: 65.6250 (71.0860)  Acc@5: 96.8750 (97.9889)
Valid: 9 [ 150/390]  Loss: 0.7969 (0.828)  Acc@1: 71.8750 (70.8920)  Acc@5: 100.0000 (98.0236)
Valid: 9 [ 200/390]  Loss: 0.7520 (0.818)  Acc@1: 71.8750 (71.2376)  Acc@5: 98.4375 (98.1266)
Valid: 9 [ 250/390]  Loss: 1.231 (0.815)  Acc@1: 59.3750 (71.3272)  Acc@5: 93.7500 (98.1511)
Valid: 9 [ 300/390]  Loss: 0.9736 (0.818)  Acc@1: 68.7500 (71.2521)  Acc@5: 98.4375 (98.1624)
Valid: 9 [ 350/390]  Loss: 1.047 (0.822)  Acc@1: 62.5000 (71.1494)  Acc@5: 96.8750 (98.0903)
Valid: 9 [ 390/390]  Loss: 0.5898 (0.816)  Acc@1: 75.0000 (71.3840)  Acc@5: 100.0000 (98.0960)
valid_acc 71.384000
epoch = 9   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3522, 0.6478],
        [0.3408, 0.6592],
        [0.3392, 0.6608],
        [0.3065, 0.6935],
        [0.3941, 0.6059],
        [0.3804, 0.6196],
        [0.3194, 0.6806],
        [0.4169, 0.5831],
        [0.4225, 0.5775],
        [0.3797, 0.6203],
        [0.3902, 0.6098],
        [0.4287, 0.5713],
        [0.4074, 0.5926],
        [0.4418, 0.5582]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3376, 0.6624],
        [0.3658, 0.6342],
        [0.3788, 0.6212],
        [0.4137, 0.5863],
        [0.4349, 0.5651],
        [0.3599, 0.6401],
        [0.3976, 0.6024],
        [0.4151, 0.5849],
        [0.4424, 0.5576],
        [0.3712, 0.6288],
        [0.4059, 0.5941],
        [0.4167, 0.5833],
        [0.4249, 0.5751],
        [0.4458, 0.5542]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 10 [   0/390]  Loss: 0.7486 (0.749)  Acc@1: 78.1250 (78.1250)  Acc@5: 96.8750 (96.8750)LR: 2.271e-02
Train: 10 [  50/390]  Loss: 0.5543 (0.704)  Acc@1: 84.3750 (75.9498)  Acc@5: 95.3125 (98.4375)LR: 2.271e-02
Train: 10 [ 100/390]  Loss: 0.7280 (0.690)  Acc@1: 70.3125 (76.5316)  Acc@5: 100.0000 (98.7160)LR: 2.271e-02
Train: 10 [ 150/390]  Loss: 0.7823 (0.695)  Acc@1: 73.4375 (75.8485)  Acc@5: 96.8750 (98.7479)LR: 2.271e-02
Train: 10 [ 200/390]  Loss: 0.5072 (0.688)  Acc@1: 84.3750 (76.1816)  Acc@5: 100.0000 (98.7718)LR: 2.271e-02
Train: 10 [ 250/390]  Loss: 0.5022 (0.690)  Acc@1: 81.2500 (76.1018)  Acc@5: 98.4375 (98.6741)LR: 2.271e-02
Train: 10 [ 300/390]  Loss: 0.4568 (0.690)  Acc@1: 87.5000 (76.1057)  Acc@5: 100.0000 (98.6244)LR: 2.271e-02
Train: 10 [ 350/390]  Loss: 0.4925 (0.685)  Acc@1: 81.2500 (76.3043)  Acc@5: 100.0000 (98.4731)LR: 2.271e-02
Train: 10 [ 390/390]  Loss: 0.6481 (0.688)  Acc@1: 80.0000 (76.1920)  Acc@5: 97.5000 (98.4440)LR: 2.271e-02
train_acc 76.192000
Valid: 10 [   0/390]  Loss: 0.7695 (0.770)  Acc@1: 76.5625 (76.5625)  Acc@5: 96.8750 (96.8750)
Valid: 10 [  50/390]  Loss: 0.7148 (0.699)  Acc@1: 70.3125 (75.1225)  Acc@5: 100.0000 (98.4988)
Valid: 10 [ 100/390]  Loss: 0.6973 (0.701)  Acc@1: 78.1250 (75.1083)  Acc@5: 98.4375 (98.3137)
Valid: 10 [ 150/390]  Loss: 0.7769 (0.707)  Acc@1: 78.1250 (75.1138)  Acc@5: 95.3125 (98.3651)
Valid: 10 [ 200/390]  Loss: 0.8462 (0.711)  Acc@1: 65.6250 (74.9378)  Acc@5: 96.8750 (98.3520)
Valid: 10 [ 250/390]  Loss: 1.049 (0.713)  Acc@1: 59.3750 (74.8630)  Acc@5: 96.8750 (98.3815)
Valid: 10 [ 300/390]  Loss: 0.7036 (0.716)  Acc@1: 70.3125 (74.7924)  Acc@5: 98.4375 (98.3441)
Valid: 10 [ 350/390]  Loss: 0.7700 (0.712)  Acc@1: 75.0000 (74.8932)  Acc@5: 98.4375 (98.3218)
Valid: 10 [ 390/390]  Loss: 0.7090 (0.711)  Acc@1: 67.5000 (74.9480)  Acc@5: 97.5000 (98.3080)
valid_acc 74.948000
epoch = 10   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3432, 0.6568],
        [0.3303, 0.6697],
        [0.3270, 0.6730],
        [0.2933, 0.7067],
        [0.3840, 0.6160],
        [0.3692, 0.6308],
        [0.3097, 0.6903],
        [0.4124, 0.5876],
        [0.4147, 0.5853],
        [0.3728, 0.6272],
        [0.3787, 0.6213],
        [0.4254, 0.5746],
        [0.4034, 0.5966],
        [0.4339, 0.5661]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3313, 0.6687],
        [0.3541, 0.6459],
        [0.3686, 0.6314],
        [0.4032, 0.5968],
        [0.4295, 0.5705],
        [0.3484, 0.6516],
        [0.3876, 0.6124],
        [0.4110, 0.5890],
        [0.4329, 0.5671],
        [0.3604, 0.6396],
        [0.3977, 0.6023],
        [0.4070, 0.5930],
        [0.4200, 0.5800],
        [0.4433, 0.5567]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 11 [   0/390]  Loss: 0.8263 (0.826)  Acc@1: 67.1875 (67.1875)  Acc@5: 93.7500 (93.7500)LR: 2.225e-02
Train: 11 [  50/390]  Loss: 0.5470 (0.635)  Acc@1: 84.3750 (77.4816)  Acc@5: 100.0000 (98.4681)LR: 2.225e-02
Train: 11 [ 100/390]  Loss: 0.5796 (0.636)  Acc@1: 81.2500 (77.8620)  Acc@5: 98.4375 (98.5767)LR: 2.225e-02
Train: 11 [ 150/390]  Loss: 0.4372 (0.631)  Acc@1: 85.9375 (78.1767)  Acc@5: 100.0000 (98.7583)LR: 2.225e-02
Train: 11 [ 200/390]  Loss: 0.6055 (0.637)  Acc@1: 84.3750 (77.9229)  Acc@5: 96.8750 (98.7407)LR: 2.225e-02
Train: 11 [ 250/390]  Loss: 0.7751 (0.642)  Acc@1: 73.4375 (77.6892)  Acc@5: 98.4375 (98.7301)LR: 2.225e-02
Train: 11 [ 300/390]  Loss: 0.9040 (0.646)  Acc@1: 70.3125 (77.5488)  Acc@5: 98.4375 (98.7282)LR: 2.225e-02
Train: 11 [ 350/390]  Loss: 0.6160 (0.643)  Acc@1: 78.1250 (77.5507)  Acc@5: 100.0000 (98.7402)LR: 2.225e-02
Train: 11 [ 390/390]  Loss: 0.5304 (0.637)  Acc@1: 77.5000 (77.8200)  Acc@5: 100.0000 (98.7920)LR: 2.225e-02
train_acc 77.820000
Valid: 11 [   0/390]  Loss: 0.7930 (0.793)  Acc@1: 68.7500 (68.7500)  Acc@5: 95.3125 (95.3125)
Valid: 11 [  50/390]  Loss: 0.8423 (0.698)  Acc@1: 73.4375 (74.2953)  Acc@5: 98.4375 (98.6213)
Valid: 11 [ 100/390]  Loss: 0.9170 (0.711)  Acc@1: 68.7500 (74.5050)  Acc@5: 98.4375 (98.4839)
Valid: 11 [ 150/390]  Loss: 0.7974 (0.705)  Acc@1: 68.7500 (74.6068)  Acc@5: 100.0000 (98.5824)
Valid: 11 [ 200/390]  Loss: 0.7373 (0.694)  Acc@1: 76.5625 (75.2099)  Acc@5: 98.4375 (98.6707)
Valid: 11 [ 250/390]  Loss: 0.6328 (0.689)  Acc@1: 76.5625 (75.5478)  Acc@5: 98.4375 (98.6990)
Valid: 11 [ 300/390]  Loss: 0.5205 (0.688)  Acc@1: 81.2500 (75.4983)  Acc@5: 100.0000 (98.7282)
Valid: 11 [ 350/390]  Loss: 0.9087 (0.691)  Acc@1: 71.8750 (75.4229)  Acc@5: 93.7500 (98.6512)
Valid: 11 [ 390/390]  Loss: 0.6650 (0.692)  Acc@1: 75.0000 (75.5240)  Acc@5: 100.0000 (98.6480)
valid_acc 75.524000
epoch = 11   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3358, 0.6642],
        [0.3205, 0.6795],
        [0.3194, 0.6806],
        [0.2838, 0.7162],
        [0.3722, 0.6278],
        [0.3623, 0.6377],
        [0.3013, 0.6987],
        [0.4073, 0.5927],
        [0.4110, 0.5890],
        [0.3660, 0.6340],
        [0.3733, 0.6267],
        [0.4248, 0.5752],
        [0.3955, 0.6045],
        [0.4246, 0.5754]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3209, 0.6791],
        [0.3468, 0.6532],
        [0.3583, 0.6417],
        [0.3979, 0.6021],
        [0.4210, 0.5790],
        [0.3381, 0.6619],
        [0.3801, 0.6199],
        [0.4087, 0.5913],
        [0.4254, 0.5746],
        [0.3504, 0.6496],
        [0.3917, 0.6083],
        [0.3990, 0.6010],
        [0.4141, 0.5859],
        [0.4352, 0.5648]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 12 [   0/390]  Loss: 0.6900 (0.690)  Acc@1: 73.4375 (73.4375)  Acc@5: 98.4375 (98.4375)LR: 2.175e-02
Train: 12 [  50/390]  Loss: 0.4630 (0.626)  Acc@1: 85.9375 (78.3088)  Acc@5: 100.0000 (98.8664)LR: 2.175e-02
Train: 12 [ 100/390]  Loss: 0.6061 (0.620)  Acc@1: 75.0000 (78.3571)  Acc@5: 98.4375 (98.8861)LR: 2.175e-02
Train: 12 [ 150/390]  Loss: 0.4541 (0.611)  Acc@1: 84.3750 (79.0046)  Acc@5: 100.0000 (98.9445)LR: 2.175e-02
Train: 12 [ 200/390]  Loss: 0.5591 (0.599)  Acc@1: 76.5625 (79.3377)  Acc@5: 100.0000 (98.9583)LR: 2.175e-02
Train: 12 [ 250/390]  Loss: 0.7172 (0.605)  Acc@1: 78.1250 (79.0027)  Acc@5: 100.0000 (98.9666)LR: 2.175e-02
Train: 12 [ 300/390]  Loss: 0.6572 (0.605)  Acc@1: 73.4375 (78.9504)  Acc@5: 98.4375 (98.9203)LR: 2.175e-02
Train: 12 [ 350/390]  Loss: 0.4420 (0.603)  Acc@1: 84.3750 (79.0554)  Acc@5: 96.8750 (98.9138)LR: 2.175e-02
Train: 12 [ 390/390]  Loss: 0.6636 (0.609)  Acc@1: 80.0000 (78.7800)  Acc@5: 100.0000 (98.9040)LR: 2.175e-02
train_acc 78.780000
Valid: 12 [   0/390]  Loss: 0.7075 (0.708)  Acc@1: 76.5625 (76.5625)  Acc@5: 100.0000 (100.0000)
Valid: 12 [  50/390]  Loss: 0.7261 (0.744)  Acc@1: 76.5625 (74.9387)  Acc@5: 96.8750 (98.1618)
Valid: 12 [ 100/390]  Loss: 0.7593 (0.737)  Acc@1: 78.1250 (74.6906)  Acc@5: 98.4375 (98.1436)
Valid: 12 [ 150/390]  Loss: 0.7583 (0.748)  Acc@1: 73.4375 (74.3171)  Acc@5: 98.4375 (98.0857)
Valid: 12 [ 200/390]  Loss: 0.8345 (0.750)  Acc@1: 68.7500 (74.4248)  Acc@5: 98.4375 (98.0333)
Valid: 12 [ 250/390]  Loss: 0.7778 (0.746)  Acc@1: 68.7500 (74.5456)  Acc@5: 96.8750 (98.0515)
Valid: 12 [ 300/390]  Loss: 0.7500 (0.753)  Acc@1: 70.3125 (74.2110)  Acc@5: 96.8750 (97.9911)
Valid: 12 [ 350/390]  Loss: 0.8174 (0.754)  Acc@1: 78.1250 (74.2477)  Acc@5: 98.4375 (97.9745)
Valid: 12 [ 390/390]  Loss: 0.9053 (0.755)  Acc@1: 67.5000 (74.2160)  Acc@5: 100.0000 (98.0000)
valid_acc 74.216000
epoch = 12   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3320, 0.6680],
        [0.3125, 0.6875],
        [0.3112, 0.6888],
        [0.2756, 0.7244],
        [0.3653, 0.6347],
        [0.3535, 0.6465],
        [0.2919, 0.7081],
        [0.4015, 0.5985],
        [0.4112, 0.5888],
        [0.3556, 0.6444],
        [0.3668, 0.6332],
        [0.4181, 0.5819],
        [0.3909, 0.6091],
        [0.4213, 0.5787]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3133, 0.6867],
        [0.3364, 0.6636],
        [0.3527, 0.6473],
        [0.3884, 0.6116],
        [0.4163, 0.5837],
        [0.3278, 0.6722],
        [0.3678, 0.6322],
        [0.4011, 0.5989],
        [0.4238, 0.5762],
        [0.3452, 0.6548],
        [0.3816, 0.6184],
        [0.3902, 0.6098],
        [0.4126, 0.5874],
        [0.4268, 0.5732]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 13 [   0/390]  Loss: 0.3947 (0.395)  Acc@1: 87.5000 (87.5000)  Acc@5: 100.0000 (100.0000)LR: 2.121e-02
Train: 13 [  50/390]  Loss: 0.4279 (0.543)  Acc@1: 84.3750 (80.8517)  Acc@5: 100.0000 (99.2341)LR: 2.121e-02
Train: 13 [ 100/390]  Loss: 0.5543 (0.560)  Acc@1: 82.8125 (80.4920)  Acc@5: 100.0000 (99.0563)LR: 2.121e-02
Train: 13 [ 150/390]  Loss: 0.7943 (0.574)  Acc@1: 73.4375 (80.0186)  Acc@5: 98.4375 (98.9756)LR: 2.121e-02
Train: 13 [ 200/390]  Loss: 0.9035 (0.575)  Acc@1: 67.1875 (79.9129)  Acc@5: 98.4375 (98.9972)LR: 2.121e-02
Train: 13 [ 250/390]  Loss: 0.6216 (0.566)  Acc@1: 81.2500 (80.2291)  Acc@5: 98.4375 (98.9729)LR: 2.121e-02
Train: 13 [ 300/390]  Loss: 0.4642 (0.571)  Acc@1: 87.5000 (80.1287)  Acc@5: 100.0000 (98.9722)LR: 2.121e-02
Train: 13 [ 350/390]  Loss: 0.4787 (0.572)  Acc@1: 82.8125 (80.1327)  Acc@5: 98.4375 (98.9672)LR: 2.121e-02
Train: 13 [ 390/390]  Loss: 0.7280 (0.577)  Acc@1: 80.0000 (80.0280)  Acc@5: 97.5000 (98.9640)LR: 2.121e-02
train_acc 80.028000
Valid: 13 [   0/390]  Loss: 0.6289 (0.629)  Acc@1: 82.8125 (82.8125)  Acc@5: 98.4375 (98.4375)
Valid: 13 [  50/390]  Loss: 0.5508 (0.670)  Acc@1: 79.6875 (76.6850)  Acc@5: 100.0000 (98.6826)
Valid: 13 [ 100/390]  Loss: 0.8423 (0.676)  Acc@1: 70.3125 (76.6089)  Acc@5: 95.3125 (98.4994)
Valid: 13 [ 150/390]  Loss: 0.7715 (0.682)  Acc@1: 78.1250 (76.2314)  Acc@5: 100.0000 (98.5306)
Valid: 13 [ 200/390]  Loss: 0.6401 (0.688)  Acc@1: 82.8125 (76.1039)  Acc@5: 100.0000 (98.4686)
Valid: 13 [ 250/390]  Loss: 0.5737 (0.680)  Acc@1: 84.3750 (76.2699)  Acc@5: 100.0000 (98.5496)
Valid: 13 [ 300/390]  Loss: 0.6025 (0.674)  Acc@1: 84.3750 (76.5262)  Acc@5: 98.4375 (98.5932)
Valid: 13 [ 350/390]  Loss: 0.6895 (0.673)  Acc@1: 78.1250 (76.5670)  Acc@5: 100.0000 (98.6334)
Valid: 13 [ 390/390]  Loss: 0.5928 (0.671)  Acc@1: 80.0000 (76.6680)  Acc@5: 100.0000 (98.6680)
valid_acc 76.668000
epoch = 13   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3263, 0.6737],
        [0.3035, 0.6965],
        [0.3033, 0.6967],
        [0.2649, 0.7351],
        [0.3541, 0.6459],
        [0.3511, 0.6489],
        [0.2861, 0.7139],
        [0.4005, 0.5995],
        [0.4150, 0.5850],
        [0.3507, 0.6493],
        [0.3640, 0.6360],
        [0.4124, 0.5876],
        [0.3842, 0.6158],
        [0.4199, 0.5801]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3065, 0.6935],
        [0.3243, 0.6757],
        [0.3478, 0.6522],
        [0.3838, 0.6162],
        [0.4084, 0.5916],
        [0.3209, 0.6791],
        [0.3539, 0.6461],
        [0.3977, 0.6023],
        [0.4160, 0.5840],
        [0.3432, 0.6568],
        [0.3719, 0.6281],
        [0.3826, 0.6174],
        [0.4093, 0.5907],
        [0.4197, 0.5803]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 14 [   0/390]  Loss: 0.5624 (0.562)  Acc@1: 73.4375 (73.4375)  Acc@5: 98.4375 (98.4375)LR: 2.065e-02
Train: 14 [  50/390]  Loss: 0.4585 (0.525)  Acc@1: 82.8125 (81.3419)  Acc@5: 98.4375 (99.1728)LR: 2.065e-02
Train: 14 [ 100/390]  Loss: 0.5759 (0.522)  Acc@1: 84.3750 (81.9616)  Acc@5: 100.0000 (99.1955)LR: 2.065e-02
Train: 14 [ 150/390]  Loss: 0.4593 (0.531)  Acc@1: 84.3750 (81.5294)  Acc@5: 98.4375 (99.1722)LR: 2.065e-02
Train: 14 [ 200/390]  Loss: 0.5883 (0.539)  Acc@1: 81.2500 (81.2345)  Acc@5: 98.4375 (99.1294)LR: 2.065e-02
Train: 14 [ 250/390]  Loss: 0.5821 (0.542)  Acc@1: 84.3750 (81.1940)  Acc@5: 96.8750 (99.1472)LR: 2.065e-02
Train: 14 [ 300/390]  Loss: 0.5300 (0.543)  Acc@1: 81.2500 (81.0943)  Acc@5: 98.4375 (99.1798)LR: 2.065e-02
Train: 14 [ 350/390]  Loss: 0.6250 (0.549)  Acc@1: 75.0000 (80.9651)  Acc@5: 100.0000 (99.1542)LR: 2.065e-02
Train: 14 [ 390/390]  Loss: 0.5128 (0.549)  Acc@1: 80.0000 (81.0160)  Acc@5: 100.0000 (99.1560)LR: 2.065e-02
train_acc 81.016000
Valid: 14 [   0/390]  Loss: 0.6157 (0.616)  Acc@1: 79.6875 (79.6875)  Acc@5: 100.0000 (100.0000)
Valid: 14 [  50/390]  Loss: 0.8115 (0.630)  Acc@1: 73.4375 (79.1054)  Acc@5: 98.4375 (98.6520)
Valid: 14 [ 100/390]  Loss: 0.6025 (0.636)  Acc@1: 78.1250 (78.3106)  Acc@5: 100.0000 (98.7160)
Valid: 14 [ 150/390]  Loss: 0.5068 (0.627)  Acc@1: 85.9375 (78.3837)  Acc@5: 100.0000 (98.7272)
Valid: 14 [ 200/390]  Loss: 0.5801 (0.631)  Acc@1: 82.8125 (78.1328)  Acc@5: 96.8750 (98.7407)
Valid: 14 [ 250/390]  Loss: 0.6997 (0.632)  Acc@1: 78.1250 (78.1125)  Acc@5: 98.4375 (98.7114)
Valid: 14 [ 300/390]  Loss: 0.8696 (0.631)  Acc@1: 68.7500 (78.1458)  Acc@5: 100.0000 (98.7386)
Valid: 14 [ 350/390]  Loss: 0.4910 (0.626)  Acc@1: 85.9375 (78.2407)  Acc@5: 100.0000 (98.7625)
Valid: 14 [ 390/390]  Loss: 0.5820 (0.623)  Acc@1: 82.5000 (78.4440)  Acc@5: 97.5000 (98.7760)
valid_acc 78.444000
epoch = 14   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3222, 0.6778],
        [0.3015, 0.6985],
        [0.2971, 0.7029],
        [0.2562, 0.7438],
        [0.3512, 0.6488],
        [0.3446, 0.6554],
        [0.2785, 0.7215],
        [0.3949, 0.6051],
        [0.4139, 0.5861],
        [0.3453, 0.6547],
        [0.3611, 0.6389],
        [0.4088, 0.5912],
        [0.3802, 0.6198],
        [0.4136, 0.5864]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.2983, 0.7017],
        [0.3172, 0.6828],
        [0.3426, 0.6574],
        [0.3793, 0.6207],
        [0.4009, 0.5991],
        [0.3099, 0.6901],
        [0.3447, 0.6553],
        [0.3925, 0.6075],
        [0.4121, 0.5879],
        [0.3356, 0.6644],
        [0.3673, 0.6327],
        [0.3759, 0.6241],
        [0.4080, 0.5920],
        [0.4161, 0.5839]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 15 [   0/390]  Loss: 0.4765 (0.476)  Acc@1: 82.8125 (82.8125)  Acc@5: 100.0000 (100.0000)LR: 2.005e-02
Train: 15 [  50/390]  Loss: 0.6150 (0.503)  Acc@1: 79.6875 (82.2917)  Acc@5: 98.4375 (99.5098)LR: 2.005e-02
Train: 15 [ 100/390]  Loss: 0.6135 (0.503)  Acc@1: 78.1250 (82.4257)  Acc@5: 100.0000 (99.4431)LR: 2.005e-02
Train: 15 [ 150/390]  Loss: 0.5349 (0.514)  Acc@1: 81.2500 (82.1813)  Acc@5: 100.0000 (99.4516)LR: 2.005e-02
Train: 15 [ 200/390]  Loss: 0.5084 (0.516)  Acc@1: 82.8125 (82.2528)  Acc@5: 98.4375 (99.3781)LR: 2.005e-02
Train: 15 [ 250/390]  Loss: 0.5829 (0.522)  Acc@1: 85.9375 (82.2460)  Acc@5: 98.4375 (99.2592)LR: 2.005e-02
Train: 15 [ 300/390]  Loss: 0.3805 (0.523)  Acc@1: 87.5000 (82.1169)  Acc@5: 100.0000 (99.2317)LR: 2.005e-02
Train: 15 [ 350/390]  Loss: 0.3457 (0.522)  Acc@1: 90.6250 (82.1982)  Acc@5: 100.0000 (99.2477)LR: 2.005e-02
Train: 15 [ 390/390]  Loss: 0.4817 (0.523)  Acc@1: 77.5000 (82.2280)  Acc@5: 97.5000 (99.2480)LR: 2.005e-02
train_acc 82.228000
Valid: 15 [   0/390]  Loss: 0.6304 (0.630)  Acc@1: 75.0000 (75.0000)  Acc@5: 100.0000 (100.0000)
Valid: 15 [  50/390]  Loss: 0.5039 (0.565)  Acc@1: 89.0625 (80.1777)  Acc@5: 98.4375 (98.8664)
Valid: 15 [ 100/390]  Loss: 0.7896 (0.587)  Acc@1: 78.1250 (79.5947)  Acc@5: 98.4375 (98.9171)
Valid: 15 [ 150/390]  Loss: 0.6274 (0.588)  Acc@1: 82.8125 (79.8117)  Acc@5: 96.8750 (98.8928)
Valid: 15 [ 200/390]  Loss: 0.6455 (0.587)  Acc@1: 79.6875 (79.7341)  Acc@5: 98.4375 (98.9117)
Valid: 15 [ 250/390]  Loss: 0.4829 (0.587)  Acc@1: 82.8125 (79.7684)  Acc@5: 98.4375 (98.8982)
Valid: 15 [ 300/390]  Loss: 0.6978 (0.590)  Acc@1: 75.0000 (79.5681)  Acc@5: 100.0000 (98.9203)
Valid: 15 [ 350/390]  Loss: 0.4983 (0.586)  Acc@1: 82.8125 (79.7943)  Acc@5: 98.4375 (98.9227)
Valid: 15 [ 390/390]  Loss: 0.7104 (0.583)  Acc@1: 72.5000 (79.8800)  Acc@5: 97.5000 (98.9360)
valid_acc 79.880000
epoch = 15   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3188, 0.6812],
        [0.2938, 0.7062],
        [0.2913, 0.7087],
        [0.2520, 0.7480],
        [0.3464, 0.6536],
        [0.3427, 0.6573],
        [0.2738, 0.7262],
        [0.3925, 0.6075],
        [0.4142, 0.5858],
        [0.3422, 0.6578],
        [0.3606, 0.6394],
        [0.4024, 0.5976],
        [0.3826, 0.6174],
        [0.4069, 0.5931]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.2902, 0.7098],
        [0.3099, 0.6901],
        [0.3362, 0.6638],
        [0.3791, 0.6209],
        [0.3975, 0.6025],
        [0.3064, 0.6936],
        [0.3340, 0.6660],
        [0.3848, 0.6152],
        [0.4100, 0.5900],
        [0.3313, 0.6687],
        [0.3563, 0.6437],
        [0.3686, 0.6314],
        [0.4058, 0.5942],
        [0.4151, 0.5849]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 16 [   0/390]  Loss: 0.7585 (0.758)  Acc@1: 76.5625 (76.5625)  Acc@5: 100.0000 (100.0000)LR: 1.943e-02
Train: 16 [  50/390]  Loss: 0.3799 (0.489)  Acc@1: 87.5000 (82.9044)  Acc@5: 100.0000 (99.4792)LR: 1.943e-02
Train: 16 [ 100/390]  Loss: 0.3520 (0.480)  Acc@1: 84.3750 (83.0446)  Acc@5: 100.0000 (99.4895)LR: 1.943e-02
Train: 16 [ 150/390]  Loss: 0.4168 (0.478)  Acc@1: 87.5000 (83.1436)  Acc@5: 100.0000 (99.4516)LR: 1.943e-02
Train: 16 [ 200/390]  Loss: 0.4823 (0.484)  Acc@1: 82.8125 (83.0768)  Acc@5: 100.0000 (99.4481)LR: 1.943e-02
Train: 16 [ 250/390]  Loss: 0.5043 (0.491)  Acc@1: 84.3750 (82.7938)  Acc@5: 100.0000 (99.3650)LR: 1.943e-02
Train: 16 [ 300/390]  Loss: 0.4077 (0.493)  Acc@1: 84.3750 (82.7917)  Acc@5: 100.0000 (99.3459)LR: 1.943e-02
Train: 16 [ 350/390]  Loss: 0.4312 (0.495)  Acc@1: 84.3750 (82.7591)  Acc@5: 98.4375 (99.3501)LR: 1.943e-02
Train: 16 [ 390/390]  Loss: 0.4931 (0.499)  Acc@1: 87.5000 (82.5880)  Acc@5: 100.0000 (99.3000)LR: 1.943e-02
train_acc 82.588000
Valid: 16 [   0/390]  Loss: 0.7192 (0.719)  Acc@1: 76.5625 (76.5625)  Acc@5: 96.8750 (96.8750)
Valid: 16 [  50/390]  Loss: 0.5122 (0.588)  Acc@1: 84.3750 (80.8211)  Acc@5: 100.0000 (98.8051)
Valid: 16 [ 100/390]  Loss: 0.6860 (0.601)  Acc@1: 71.8750 (79.6720)  Acc@5: 100.0000 (98.8397)
Valid: 16 [ 150/390]  Loss: 0.5806 (0.591)  Acc@1: 81.2500 (80.0704)  Acc@5: 95.3125 (98.8307)
Valid: 16 [ 200/390]  Loss: 0.6108 (0.582)  Acc@1: 78.1250 (80.2317)  Acc@5: 100.0000 (98.9117)
Valid: 16 [ 250/390]  Loss: 0.5928 (0.580)  Acc@1: 78.1250 (80.2913)  Acc@5: 98.4375 (98.8608)
Valid: 16 [ 300/390]  Loss: 0.7378 (0.581)  Acc@1: 71.8750 (80.3000)  Acc@5: 96.8750 (98.7542)
Valid: 16 [ 350/390]  Loss: 0.6934 (0.586)  Acc@1: 78.1250 (80.1238)  Acc@5: 100.0000 (98.7491)
Valid: 16 [ 390/390]  Loss: 0.7002 (0.586)  Acc@1: 70.0000 (80.1520)  Acc@5: 100.0000 (98.7160)
valid_acc 80.152000
epoch = 16   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3136, 0.6864],
        [0.2929, 0.7071],
        [0.2904, 0.7096],
        [0.2447, 0.7553],
        [0.3462, 0.6538],
        [0.3400, 0.6600],
        [0.2673, 0.7327],
        [0.3953, 0.6047],
        [0.4179, 0.5821],
        [0.3394, 0.6606],
        [0.3581, 0.6419],
        [0.4053, 0.5947],
        [0.3814, 0.6186],
        [0.4057, 0.5943]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.2878, 0.7122],
        [0.2995, 0.7005],
        [0.3358, 0.6642],
        [0.3702, 0.6298],
        [0.3923, 0.6077],
        [0.2982, 0.7018],
        [0.3245, 0.6755],
        [0.3790, 0.6210],
        [0.4110, 0.5890],
        [0.3283, 0.6717],
        [0.3477, 0.6523],
        [0.3632, 0.6368],
        [0.4032, 0.5968],
        [0.4094, 0.5906]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 17 [   0/390]  Loss: 0.3941 (0.394)  Acc@1: 89.0625 (89.0625)  Acc@5: 100.0000 (100.0000)LR: 1.878e-02
Train: 17 [  50/390]  Loss: 0.3409 (0.476)  Acc@1: 89.0625 (83.9461)  Acc@5: 98.4375 (99.2953)LR: 1.878e-02
Train: 17 [ 100/390]  Loss: 0.4099 (0.454)  Acc@1: 89.0625 (84.5606)  Acc@5: 100.0000 (99.4121)LR: 1.878e-02
Train: 17 [ 150/390]  Loss: 0.7099 (0.463)  Acc@1: 78.1250 (84.0646)  Acc@5: 96.8750 (99.3998)LR: 1.878e-02
Train: 17 [ 200/390]  Loss: 0.2950 (0.476)  Acc@1: 89.0625 (83.6287)  Acc@5: 100.0000 (99.2771)LR: 1.878e-02
Train: 17 [ 250/390]  Loss: 0.6629 (0.470)  Acc@1: 76.5625 (83.6591)  Acc@5: 98.4375 (99.3277)LR: 1.878e-02
Train: 17 [ 300/390]  Loss: 0.5821 (0.480)  Acc@1: 79.6875 (83.3472)  Acc@5: 100.0000 (99.3044)LR: 1.878e-02
Train: 17 [ 350/390]  Loss: 0.5665 (0.483)  Acc@1: 79.6875 (83.2888)  Acc@5: 98.4375 (99.2877)LR: 1.878e-02
Train: 17 [ 390/390]  Loss: 0.5991 (0.483)  Acc@1: 75.0000 (83.2840)  Acc@5: 97.5000 (99.2680)LR: 1.878e-02
train_acc 83.284000
Valid: 17 [   0/390]  Loss: 0.5342 (0.534)  Acc@1: 82.8125 (82.8125)  Acc@5: 100.0000 (100.0000)
Valid: 17 [  50/390]  Loss: 0.4697 (0.524)  Acc@1: 82.8125 (82.1691)  Acc@5: 100.0000 (99.2034)
Valid: 17 [ 100/390]  Loss: 0.3857 (0.520)  Acc@1: 90.6250 (82.3175)  Acc@5: 98.4375 (99.1491)
Valid: 17 [ 150/390]  Loss: 0.4763 (0.521)  Acc@1: 81.2500 (82.3986)  Acc@5: 100.0000 (99.0998)
Valid: 17 [ 200/390]  Loss: 0.5195 (0.522)  Acc@1: 82.8125 (82.2606)  Acc@5: 100.0000 (99.1527)
Valid: 17 [ 250/390]  Loss: 0.8613 (0.525)  Acc@1: 73.4375 (82.1402)  Acc@5: 95.3125 (99.0787)
Valid: 17 [ 300/390]  Loss: 0.4414 (0.524)  Acc@1: 85.9375 (82.1065)  Acc@5: 98.4375 (99.1020)
Valid: 17 [ 350/390]  Loss: 0.4319 (0.523)  Acc@1: 89.0625 (82.0513)  Acc@5: 98.4375 (99.1319)
Valid: 17 [ 390/390]  Loss: 0.7144 (0.524)  Acc@1: 75.0000 (81.9240)  Acc@5: 100.0000 (99.1520)
valid_acc 81.924000
epoch = 17   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3045, 0.6955],
        [0.2927, 0.7073],
        [0.2844, 0.7156],
        [0.2404, 0.7596],
        [0.3401, 0.6599],
        [0.3353, 0.6647],
        [0.2646, 0.7354],
        [0.3918, 0.6082],
        [0.4165, 0.5835],
        [0.3345, 0.6655],
        [0.3543, 0.6457],
        [0.3999, 0.6001],
        [0.3817, 0.6183],
        [0.4009, 0.5991]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.2776, 0.7224],
        [0.2947, 0.7053],
        [0.3311, 0.6689],
        [0.3665, 0.6335],
        [0.3880, 0.6120],
        [0.2905, 0.7095],
        [0.3175, 0.6825],
        [0.3777, 0.6223],
        [0.4086, 0.5914],
        [0.3230, 0.6770],
        [0.3386, 0.6614],
        [0.3634, 0.6366],
        [0.4027, 0.5973],
        [0.4084, 0.5916]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 18 [   0/390]  Loss: 0.4486 (0.449)  Acc@1: 81.2500 (81.2500)  Acc@5: 100.0000 (100.0000)LR: 1.811e-02
Train: 18 [  50/390]  Loss: 0.3424 (0.470)  Acc@1: 87.5000 (84.0993)  Acc@5: 98.4375 (99.2341)LR: 1.811e-02
Train: 18 [ 100/390]  Loss: 0.4845 (0.470)  Acc@1: 79.6875 (83.8490)  Acc@5: 98.4375 (99.3193)LR: 1.811e-02
Train: 18 [ 150/390]  Loss: 0.6550 (0.469)  Acc@1: 73.4375 (83.8990)  Acc@5: 100.0000 (99.2757)LR: 1.811e-02
Train: 18 [ 200/390]  Loss: 0.5085 (0.466)  Acc@1: 82.8125 (83.9941)  Acc@5: 100.0000 (99.3159)LR: 1.811e-02
Train: 18 [ 250/390]  Loss: 0.4915 (0.462)  Acc@1: 85.9375 (84.1198)  Acc@5: 96.8750 (99.3650)LR: 1.811e-02
Train: 18 [ 300/390]  Loss: 0.3261 (0.463)  Acc@1: 89.0625 (84.0947)  Acc@5: 100.0000 (99.3615)LR: 1.811e-02
Train: 18 [ 350/390]  Loss: 0.2936 (0.463)  Acc@1: 92.1875 (84.0723)  Acc@5: 100.0000 (99.3857)LR: 1.811e-02
Train: 18 [ 390/390]  Loss: 0.3683 (0.461)  Acc@1: 80.0000 (84.0520)  Acc@5: 100.0000 (99.4000)LR: 1.811e-02
train_acc 84.052000
Valid: 18 [   0/390]  Loss: 0.5645 (0.564)  Acc@1: 82.8125 (82.8125)  Acc@5: 100.0000 (100.0000)
Valid: 18 [  50/390]  Loss: 0.6128 (0.537)  Acc@1: 79.6875 (80.8211)  Acc@5: 100.0000 (99.4179)
Valid: 18 [ 100/390]  Loss: 0.4412 (0.528)  Acc@1: 81.2500 (81.2036)  Acc@5: 100.0000 (99.2884)
Valid: 18 [ 150/390]  Loss: 0.4556 (0.530)  Acc@1: 84.3750 (81.4880)  Acc@5: 98.4375 (99.2136)
Valid: 18 [ 200/390]  Loss: 0.3650 (0.533)  Acc@1: 84.3750 (81.2966)  Acc@5: 100.0000 (99.1449)
Valid: 18 [ 250/390]  Loss: 0.4419 (0.526)  Acc@1: 82.8125 (81.5301)  Acc@5: 100.0000 (99.1970)
Valid: 18 [ 300/390]  Loss: 0.4443 (0.532)  Acc@1: 84.3750 (81.4317)  Acc@5: 100.0000 (99.1591)
Valid: 18 [ 350/390]  Loss: 0.4160 (0.534)  Acc@1: 84.3750 (81.3746)  Acc@5: 100.0000 (99.1453)
Valid: 18 [ 390/390]  Loss: 0.5938 (0.535)  Acc@1: 85.0000 (81.2760)  Acc@5: 100.0000 (99.1440)
valid_acc 81.276000
epoch = 18   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3027, 0.6973],
        [0.2874, 0.7126],
        [0.2828, 0.7172],
        [0.2370, 0.7630],
        [0.3338, 0.6662],
        [0.3313, 0.6687],
        [0.2643, 0.7357],
        [0.3946, 0.6054],
        [0.4147, 0.5853],
        [0.3338, 0.6662],
        [0.3480, 0.6520],
        [0.3993, 0.6007],
        [0.3792, 0.6208],
        [0.3954, 0.6046]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.2734, 0.7266],
        [0.2878, 0.7122],
        [0.3270, 0.6730],
        [0.3601, 0.6399],
        [0.3846, 0.6154],
        [0.2797, 0.7203],
        [0.3081, 0.6919],
        [0.3720, 0.6280],
        [0.4056, 0.5944],
        [0.3216, 0.6784],
        [0.3361, 0.6639],
        [0.3640, 0.6360],
        [0.4004, 0.5996],
        [0.4026, 0.5974]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 19 [   0/390]  Loss: 0.3907 (0.391)  Acc@1: 84.3750 (84.3750)  Acc@5: 100.0000 (100.0000)LR: 1.742e-02
Train: 19 [  50/390]  Loss: 0.2269 (0.439)  Acc@1: 95.3125 (84.8039)  Acc@5: 100.0000 (99.3873)LR: 1.742e-02
Train: 19 [ 100/390]  Loss: 0.4894 (0.428)  Acc@1: 82.8125 (85.0402)  Acc@5: 100.0000 (99.5359)LR: 1.742e-02
Train: 19 [ 150/390]  Loss: 0.5015 (0.426)  Acc@1: 79.6875 (84.8717)  Acc@5: 100.0000 (99.4826)LR: 1.742e-02
Train: 19 [ 200/390]  Loss: 0.4869 (0.433)  Acc@1: 84.3750 (84.8725)  Acc@5: 98.4375 (99.4714)LR: 1.742e-02
Train: 19 [ 250/390]  Loss: 0.5681 (0.433)  Acc@1: 81.2500 (84.9477)  Acc@5: 100.0000 (99.4646)LR: 1.742e-02
Train: 19 [ 300/390]  Loss: 0.3850 (0.435)  Acc@1: 84.3750 (84.8059)  Acc@5: 100.0000 (99.4446)LR: 1.742e-02
Train: 19 [ 350/390]  Loss: 0.6006 (0.437)  Acc@1: 79.6875 (84.7400)  Acc@5: 100.0000 (99.4168)LR: 1.742e-02
Train: 19 [ 390/390]  Loss: 0.5716 (0.438)  Acc@1: 72.5000 (84.6200)  Acc@5: 100.0000 (99.4200)LR: 1.742e-02
train_acc 84.620000
Valid: 19 [   0/390]  Loss: 0.4590 (0.459)  Acc@1: 81.2500 (81.2500)  Acc@5: 98.4375 (98.4375)
Valid: 19 [  50/390]  Loss: 0.6582 (0.512)  Acc@1: 79.6875 (81.8627)  Acc@5: 98.4375 (98.9277)
Valid: 19 [ 100/390]  Loss: 0.3835 (0.505)  Acc@1: 85.9375 (81.8069)  Acc@5: 98.4375 (99.1027)
Valid: 19 [ 150/390]  Loss: 0.5249 (0.512)  Acc@1: 85.9375 (81.7363)  Acc@5: 100.0000 (99.1101)
Valid: 19 [ 200/390]  Loss: 0.5137 (0.514)  Acc@1: 84.3750 (81.9263)  Acc@5: 100.0000 (99.1371)
Valid: 19 [ 250/390]  Loss: 0.3477 (0.512)  Acc@1: 89.0625 (82.1340)  Acc@5: 100.0000 (99.1409)
Valid: 19 [ 300/390]  Loss: 0.5059 (0.512)  Acc@1: 85.9375 (82.1169)  Acc@5: 100.0000 (99.1694)
Valid: 19 [ 350/390]  Loss: 0.4131 (0.511)  Acc@1: 87.5000 (82.1982)  Acc@5: 98.4375 (99.1720)
Valid: 19 [ 390/390]  Loss: 0.5420 (0.517)  Acc@1: 80.0000 (82.1080)  Acc@5: 100.0000 (99.1240)
valid_acc 82.108000
epoch = 19   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3037, 0.6963],
        [0.2897, 0.7103],
        [0.2805, 0.7195],
        [0.2344, 0.7656],
        [0.3303, 0.6697],
        [0.3265, 0.6735],
        [0.2672, 0.7328],
        [0.4005, 0.5995],
        [0.4157, 0.5843],
        [0.3302, 0.6698],
        [0.3454, 0.6546],
        [0.3944, 0.6056],
        [0.3815, 0.6185],
        [0.3890, 0.6110]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.2676, 0.7324],
        [0.2819, 0.7181],
        [0.3242, 0.6758],
        [0.3535, 0.6465],
        [0.3775, 0.6225],
        [0.2688, 0.7312],
        [0.3068, 0.6932],
        [0.3719, 0.6281],
        [0.4043, 0.5957],
        [0.3143, 0.6857],
        [0.3341, 0.6659],
        [0.3629, 0.6371],
        [0.3933, 0.6067],
        [0.4022, 0.5978]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 20 [   0/390]  Loss: 0.3820 (0.382)  Acc@1: 89.0625 (89.0625)  Acc@5: 100.0000 (100.0000)LR: 1.671e-02
Train: 20 [  50/390]  Loss: 0.2280 (0.382)  Acc@1: 90.6250 (86.9485)  Acc@5: 100.0000 (99.6630)LR: 1.671e-02
Train: 20 [ 100/390]  Loss: 0.4643 (0.404)  Acc@1: 85.9375 (86.0149)  Acc@5: 100.0000 (99.5668)LR: 1.671e-02
Train: 20 [ 150/390]  Loss: 0.2953 (0.411)  Acc@1: 87.5000 (85.7305)  Acc@5: 100.0000 (99.5757)LR: 1.671e-02
Train: 20 [ 200/390]  Loss: 0.3822 (0.414)  Acc@1: 87.5000 (85.7276)  Acc@5: 98.4375 (99.5569)LR: 1.671e-02
Train: 20 [ 250/390]  Loss: 0.3683 (0.415)  Acc@1: 87.5000 (85.6449)  Acc@5: 100.0000 (99.5331)LR: 1.671e-02
Train: 20 [ 300/390]  Loss: 0.6894 (0.420)  Acc@1: 78.1250 (85.6987)  Acc@5: 96.8750 (99.4913)LR: 1.671e-02
Train: 20 [ 350/390]  Loss: 0.3745 (0.418)  Acc@1: 85.9375 (85.6927)  Acc@5: 100.0000 (99.4881)LR: 1.671e-02
Train: 20 [ 390/390]  Loss: 0.5428 (0.418)  Acc@1: 85.0000 (85.7080)  Acc@5: 97.5000 (99.4760)LR: 1.671e-02
train_acc 85.708000
Valid: 20 [   0/390]  Loss: 0.6245 (0.625)  Acc@1: 78.1250 (78.1250)  Acc@5: 98.4375 (98.4375)
Valid: 20 [  50/390]  Loss: 0.5811 (0.537)  Acc@1: 76.5625 (80.8517)  Acc@5: 100.0000 (99.3566)
Valid: 20 [ 100/390]  Loss: 0.5171 (0.562)  Acc@1: 85.9375 (80.6776)  Acc@5: 96.8750 (99.1027)
Valid: 20 [ 150/390]  Loss: 0.5747 (0.562)  Acc@1: 75.0000 (80.4843)  Acc@5: 96.8750 (99.0998)
Valid: 20 [ 200/390]  Loss: 0.4231 (0.560)  Acc@1: 85.9375 (80.4649)  Acc@5: 98.4375 (99.0905)
Valid: 20 [ 250/390]  Loss: 0.7646 (0.556)  Acc@1: 68.7500 (80.7582)  Acc@5: 100.0000 (99.1223)
Valid: 20 [ 300/390]  Loss: 0.8599 (0.559)  Acc@1: 71.8750 (80.6842)  Acc@5: 96.8750 (99.1123)
Valid: 20 [ 350/390]  Loss: 0.6841 (0.559)  Acc@1: 78.1250 (80.6980)  Acc@5: 100.0000 (99.1319)
Valid: 20 [ 390/390]  Loss: 0.7632 (0.560)  Acc@1: 75.0000 (80.6520)  Acc@5: 100.0000 (99.1360)
valid_acc 80.652000
epoch = 20   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3040, 0.6960],
        [0.2870, 0.7130],
        [0.2772, 0.7228],
        [0.2313, 0.7687],
        [0.3287, 0.6713],
        [0.3285, 0.6715],
        [0.2649, 0.7351],
        [0.3980, 0.6020],
        [0.4138, 0.5862],
        [0.3275, 0.6725],
        [0.3412, 0.6588],
        [0.3940, 0.6060],
        [0.3835, 0.6165],
        [0.3881, 0.6119]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.2615, 0.7385],
        [0.2808, 0.7192],
        [0.3191, 0.6809],
        [0.3494, 0.6506],
        [0.3736, 0.6264],
        [0.2657, 0.7343],
        [0.3036, 0.6964],
        [0.3696, 0.6304],
        [0.4045, 0.5955],
        [0.3083, 0.6917],
        [0.3319, 0.6681],
        [0.3631, 0.6369],
        [0.3957, 0.6043],
        [0.3968, 0.6032]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 21 [   0/390]  Loss: 0.1674 (0.167)  Acc@1: 95.3125 (95.3125)  Acc@5: 100.0000 (100.0000)LR: 1.598e-02
Train: 21 [  50/390]  Loss: 0.4216 (0.362)  Acc@1: 87.5000 (86.7647)  Acc@5: 100.0000 (99.6936)LR: 1.598e-02
Train: 21 [ 100/390]  Loss: 0.4681 (0.381)  Acc@1: 81.2500 (85.9839)  Acc@5: 100.0000 (99.6751)LR: 1.598e-02
Train: 21 [ 150/390]  Loss: 0.3557 (0.376)  Acc@1: 85.9375 (86.3928)  Acc@5: 100.0000 (99.6585)LR: 1.598e-02
Train: 21 [ 200/390]  Loss: 0.3191 (0.385)  Acc@1: 92.1875 (86.2484)  Acc@5: 100.0000 (99.5958)LR: 1.598e-02
Train: 21 [ 250/390]  Loss: 0.3920 (0.392)  Acc@1: 82.8125 (86.0496)  Acc@5: 100.0000 (99.5829)LR: 1.598e-02
Train: 21 [ 300/390]  Loss: 0.2841 (0.394)  Acc@1: 87.5000 (86.0465)  Acc@5: 100.0000 (99.5640)LR: 1.598e-02
Train: 21 [ 350/390]  Loss: 0.5133 (0.405)  Acc@1: 79.6875 (85.6838)  Acc@5: 98.4375 (99.5326)LR: 1.598e-02
Train: 21 [ 390/390]  Loss: 0.3022 (0.404)  Acc@1: 95.0000 (85.6760)  Acc@5: 100.0000 (99.5320)LR: 1.598e-02
train_acc 85.676000
Valid: 21 [   0/390]  Loss: 0.5767 (0.577)  Acc@1: 73.4375 (73.4375)  Acc@5: 98.4375 (98.4375)
Valid: 21 [  50/390]  Loss: 0.7153 (0.513)  Acc@1: 75.0000 (82.7206)  Acc@5: 98.4375 (99.0502)
Valid: 21 [ 100/390]  Loss: 0.5684 (0.526)  Acc@1: 81.2500 (82.1627)  Acc@5: 98.4375 (98.9944)
Valid: 21 [ 150/390]  Loss: 0.6880 (0.533)  Acc@1: 76.5625 (81.9123)  Acc@5: 100.0000 (99.0377)
Valid: 21 [ 200/390]  Loss: 0.7690 (0.525)  Acc@1: 76.5625 (82.2606)  Acc@5: 95.3125 (99.0283)
Valid: 21 [ 250/390]  Loss: 0.4736 (0.518)  Acc@1: 81.2500 (82.4016)  Acc@5: 100.0000 (99.0725)
Valid: 21 [ 300/390]  Loss: 0.8491 (0.515)  Acc@1: 79.6875 (82.5322)  Acc@5: 98.4375 (99.0760)
Valid: 21 [ 350/390]  Loss: 0.4028 (0.513)  Acc@1: 89.0625 (82.6122)  Acc@5: 100.0000 (99.0696)
Valid: 21 [ 390/390]  Loss: 0.3223 (0.511)  Acc@1: 90.0000 (82.5880)  Acc@5: 97.5000 (99.1080)
valid_acc 82.588000
epoch = 21   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3052, 0.6948],
        [0.2853, 0.7147],
        [0.2751, 0.7249],
        [0.2312, 0.7688],
        [0.3231, 0.6769],
        [0.3330, 0.6670],
        [0.2648, 0.7352],
        [0.3954, 0.6046],
        [0.4133, 0.5867],
        [0.3266, 0.6734],
        [0.3426, 0.6574],
        [0.3940, 0.6060],
        [0.3826, 0.6174],
        [0.3820, 0.6180]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.2572, 0.7428],
        [0.2759, 0.7241],
        [0.3128, 0.6872],
        [0.3453, 0.6547],
        [0.3729, 0.6271],
        [0.2597, 0.7403],
        [0.3000, 0.7000],
        [0.3666, 0.6334],
        [0.3986, 0.6014],
        [0.3040, 0.6960],
        [0.3331, 0.6669],
        [0.3608, 0.6392],
        [0.3940, 0.6060],
        [0.3906, 0.6094]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 22 [   0/390]  Loss: 0.2878 (0.288)  Acc@1: 92.1875 (92.1875)  Acc@5: 100.0000 (100.0000)LR: 1.525e-02
Train: 22 [  50/390]  Loss: 0.4478 (0.392)  Acc@1: 82.8125 (86.5502)  Acc@5: 100.0000 (99.6324)LR: 1.525e-02
Train: 22 [ 100/390]  Loss: 0.4671 (0.384)  Acc@1: 84.3750 (86.7884)  Acc@5: 100.0000 (99.5668)LR: 1.525e-02
Train: 22 [ 150/390]  Loss: 0.2774 (0.379)  Acc@1: 89.0625 (86.8171)  Acc@5: 100.0000 (99.5757)LR: 1.525e-02
Train: 22 [ 200/390]  Loss: 0.3890 (0.380)  Acc@1: 84.3750 (86.5516)  Acc@5: 100.0000 (99.6035)LR: 1.525e-02
Train: 22 [ 250/390]  Loss: 0.4733 (0.384)  Acc@1: 89.0625 (86.4604)  Acc@5: 98.4375 (99.5705)LR: 1.525e-02
Train: 22 [ 300/390]  Loss: 0.4682 (0.380)  Acc@1: 81.2500 (86.5760)  Acc@5: 98.4375 (99.5743)LR: 1.525e-02
Train: 22 [ 350/390]  Loss: 0.4268 (0.383)  Acc@1: 87.5000 (86.4583)  Acc@5: 100.0000 (99.5593)LR: 1.525e-02
Train: 22 [ 390/390]  Loss: 0.2276 (0.384)  Acc@1: 90.0000 (86.4800)  Acc@5: 100.0000 (99.5600)LR: 1.525e-02
train_acc 86.480000
Valid: 22 [   0/390]  Loss: 0.7383 (0.738)  Acc@1: 70.3125 (70.3125)  Acc@5: 96.8750 (96.8750)
Valid: 22 [  50/390]  Loss: 0.6914 (0.516)  Acc@1: 75.0000 (81.7096)  Acc@5: 98.4375 (99.3873)
Valid: 22 [ 100/390]  Loss: 0.6313 (0.524)  Acc@1: 78.1250 (81.4047)  Acc@5: 96.8750 (99.3038)
Valid: 22 [ 150/390]  Loss: 0.3232 (0.513)  Acc@1: 89.0625 (81.7674)  Acc@5: 100.0000 (99.2964)
Valid: 22 [ 200/390]  Loss: 0.4602 (0.518)  Acc@1: 78.1250 (81.7009)  Acc@5: 100.0000 (99.2304)
Valid: 22 [ 250/390]  Loss: 0.4380 (0.515)  Acc@1: 90.6250 (81.8850)  Acc@5: 100.0000 (99.2281)
Valid: 22 [ 300/390]  Loss: 0.3918 (0.517)  Acc@1: 89.0625 (81.7120)  Acc@5: 100.0000 (99.2058)
Valid: 22 [ 350/390]  Loss: 0.5054 (0.519)  Acc@1: 82.8125 (81.7886)  Acc@5: 100.0000 (99.1987)
Valid: 22 [ 390/390]  Loss: 0.4934 (0.515)  Acc@1: 85.0000 (82.0000)  Acc@5: 97.5000 (99.1840)
valid_acc 82.000000
epoch = 22   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3009, 0.6991],
        [0.2862, 0.7138],
        [0.2775, 0.7225],
        [0.2341, 0.7659],
        [0.3216, 0.6784],
        [0.3321, 0.6679],
        [0.2658, 0.7342],
        [0.3957, 0.6043],
        [0.4179, 0.5821],
        [0.3230, 0.6770],
        [0.3437, 0.6563],
        [0.3960, 0.6040],
        [0.3852, 0.6148],
        [0.3833, 0.6167]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.2527, 0.7473],
        [0.2746, 0.7254],
        [0.3092, 0.6908],
        [0.3347, 0.6653],
        [0.3769, 0.6231],
        [0.2552, 0.7448],
        [0.2883, 0.7117],
        [0.3669, 0.6331],
        [0.3927, 0.6073],
        [0.2963, 0.7037],
        [0.3231, 0.6769],
        [0.3621, 0.6379],
        [0.3940, 0.6060],
        [0.3890, 0.6110]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 23 [   0/390]  Loss: 0.5853 (0.585)  Acc@1: 79.6875 (79.6875)  Acc@5: 100.0000 (100.0000)LR: 1.450e-02
Train: 23 [  50/390]  Loss: 0.3607 (0.366)  Acc@1: 89.0625 (87.3468)  Acc@5: 100.0000 (99.6936)LR: 1.450e-02
Train: 23 [ 100/390]  Loss: 0.5727 (0.369)  Acc@1: 78.1250 (87.2215)  Acc@5: 100.0000 (99.6287)LR: 1.450e-02
Train: 23 [ 150/390]  Loss: 0.3797 (0.368)  Acc@1: 89.0625 (87.2517)  Acc@5: 98.4375 (99.6171)LR: 1.450e-02
Train: 23 [ 200/390]  Loss: 0.5812 (0.365)  Acc@1: 71.8750 (87.3290)  Acc@5: 98.4375 (99.6035)LR: 1.450e-02
Train: 23 [ 250/390]  Loss: 0.3939 (0.365)  Acc@1: 87.5000 (87.3568)  Acc@5: 100.0000 (99.5829)LR: 1.450e-02
Train: 23 [ 300/390]  Loss: 0.3342 (0.365)  Acc@1: 89.0625 (87.3858)  Acc@5: 100.0000 (99.5951)LR: 1.450e-02
Train: 23 [ 350/390]  Loss: 0.2879 (0.368)  Acc@1: 89.0625 (87.3041)  Acc@5: 100.0000 (99.6172)LR: 1.450e-02
Train: 23 [ 390/390]  Loss: 0.3706 (0.371)  Acc@1: 85.0000 (87.2120)  Acc@5: 100.0000 (99.5920)LR: 1.450e-02
train_acc 87.212000
Valid: 23 [   0/390]  Loss: 0.3350 (0.335)  Acc@1: 90.6250 (90.6250)  Acc@5: 100.0000 (100.0000)
Valid: 23 [  50/390]  Loss: 0.3296 (0.572)  Acc@1: 84.3750 (80.3922)  Acc@5: 100.0000 (99.1115)
Valid: 23 [ 100/390]  Loss: 0.4409 (0.558)  Acc@1: 89.0625 (80.6776)  Acc@5: 100.0000 (99.1646)
Valid: 23 [ 150/390]  Loss: 0.6470 (0.545)  Acc@1: 76.5625 (81.1983)  Acc@5: 98.4375 (99.1618)
Valid: 23 [ 200/390]  Loss: 0.8843 (0.545)  Acc@1: 75.0000 (81.3200)  Acc@5: 96.8750 (99.1527)
Valid: 23 [ 250/390]  Loss: 0.6641 (0.548)  Acc@1: 71.8750 (81.2500)  Acc@5: 98.4375 (99.1970)
Valid: 23 [ 300/390]  Loss: 0.7129 (0.549)  Acc@1: 81.2500 (81.2760)  Acc@5: 98.4375 (99.2213)
Valid: 23 [ 350/390]  Loss: 0.5298 (0.547)  Acc@1: 81.2500 (81.3390)  Acc@5: 100.0000 (99.2254)
Valid: 23 [ 390/390]  Loss: 0.5815 (0.547)  Acc@1: 82.5000 (81.3200)  Acc@5: 100.0000 (99.2080)
valid_acc 81.320000
epoch = 23   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.2983, 0.7017],
        [0.2844, 0.7156],
        [0.2762, 0.7238],
        [0.2325, 0.7675],
        [0.3164, 0.6836],
        [0.3359, 0.6641],
        [0.2685, 0.7315],
        [0.3969, 0.6031],
        [0.4149, 0.5851],
        [0.3286, 0.6714],
        [0.3479, 0.6521],
        [0.3999, 0.6001],
        [0.3927, 0.6073],
        [0.3851, 0.6149]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.2465, 0.7535],
        [0.2699, 0.7301],
        [0.3076, 0.6924],
        [0.3269, 0.6731],
        [0.3706, 0.6294],
        [0.2500, 0.7500],
        [0.2846, 0.7154],
        [0.3627, 0.6373],
        [0.3872, 0.6128],
        [0.2950, 0.7050],
        [0.3161, 0.6839],
        [0.3594, 0.6406],
        [0.3930, 0.6070],
        [0.3917, 0.6083]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 24 [   0/390]  Loss: 0.4667 (0.467)  Acc@1: 84.3750 (84.3750)  Acc@5: 98.4375 (98.4375)LR: 1.375e-02
Train: 24 [  50/390]  Loss: 0.3270 (0.359)  Acc@1: 92.1875 (87.2549)  Acc@5: 96.8750 (99.5098)LR: 1.375e-02
Train: 24 [ 100/390]  Loss: 0.3286 (0.353)  Acc@1: 87.5000 (87.5928)  Acc@5: 100.0000 (99.5514)LR: 1.375e-02
Train: 24 [ 150/390]  Loss: 0.3819 (0.363)  Acc@1: 85.9375 (87.1378)  Acc@5: 98.4375 (99.5447)LR: 1.375e-02
Train: 24 [ 200/390]  Loss: 0.8415 (0.365)  Acc@1: 75.0000 (87.2823)  Acc@5: 96.8750 (99.5336)LR: 1.375e-02
Train: 24 [ 250/390]  Loss: 0.2913 (0.364)  Acc@1: 89.0625 (87.1016)  Acc@5: 100.0000 (99.5269)LR: 1.375e-02
Train: 24 [ 300/390]  Loss: 0.3941 (0.362)  Acc@1: 87.5000 (87.1782)  Acc@5: 100.0000 (99.5691)LR: 1.375e-02
Train: 24 [ 350/390]  Loss: 0.2914 (0.362)  Acc@1: 87.5000 (87.1617)  Acc@5: 100.0000 (99.5949)LR: 1.375e-02
Train: 24 [ 390/390]  Loss: 0.2885 (0.360)  Acc@1: 87.5000 (87.2640)  Acc@5: 100.0000 (99.5960)LR: 1.375e-02
train_acc 87.264000
Valid: 24 [   0/390]  Loss: 0.5186 (0.519)  Acc@1: 79.6875 (79.6875)  Acc@5: 98.4375 (98.4375)
Valid: 24 [  50/390]  Loss: 0.4724 (0.533)  Acc@1: 79.6875 (82.1998)  Acc@5: 100.0000 (99.0196)
Valid: 24 [ 100/390]  Loss: 0.8975 (0.538)  Acc@1: 81.2500 (82.0545)  Acc@5: 98.4375 (99.1027)
Valid: 24 [ 150/390]  Loss: 0.5117 (0.531)  Acc@1: 79.6875 (82.1709)  Acc@5: 98.4375 (99.0584)
Valid: 24 [ 200/390]  Loss: 0.3740 (0.532)  Acc@1: 87.5000 (82.1595)  Acc@5: 100.0000 (99.0905)
Valid: 24 [ 250/390]  Loss: 0.5059 (0.535)  Acc@1: 81.2500 (82.0468)  Acc@5: 100.0000 (99.1098)
Valid: 24 [ 300/390]  Loss: 0.5229 (0.531)  Acc@1: 84.3750 (82.2363)  Acc@5: 100.0000 (99.0968)
Valid: 24 [ 350/390]  Loss: 0.5454 (0.529)  Acc@1: 71.8750 (82.3851)  Acc@5: 100.0000 (99.0696)
Valid: 24 [ 390/390]  Loss: 0.4685 (0.530)  Acc@1: 80.0000 (82.3560)  Acc@5: 100.0000 (99.0720)
valid_acc 82.356000
epoch = 24   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.2993, 0.7007],
        [0.2836, 0.7164],
        [0.2770, 0.7230],
        [0.2285, 0.7715],
        [0.3145, 0.6855],
        [0.3412, 0.6588],
        [0.2693, 0.7307],
        [0.4063, 0.5937],
        [0.4207, 0.5793],
        [0.3291, 0.6709],
        [0.3504, 0.6496],
        [0.4003, 0.5997],
        [0.3950, 0.6050],
        [0.3823, 0.6177]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.2460, 0.7540],
        [0.2599, 0.7401],
        [0.3069, 0.6931],
        [0.3219, 0.6781],
        [0.3713, 0.6287],
        [0.2428, 0.7572],
        [0.2768, 0.7232],
        [0.3578, 0.6422],
        [0.3828, 0.6172],
        [0.2933, 0.7067],
        [0.3122, 0.6878],
        [0.3615, 0.6385],
        [0.3890, 0.6110],
        [0.3879, 0.6121]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 25 [   0/390]  Loss: 0.3978 (0.398)  Acc@1: 87.5000 (87.5000)  Acc@5: 100.0000 (100.0000)LR: 1.300e-02
Train: 25 [  50/390]  Loss: 0.2827 (0.323)  Acc@1: 92.1875 (88.6949)  Acc@5: 100.0000 (99.7855)LR: 1.300e-02
Train: 25 [ 100/390]  Loss: 0.2380 (0.338)  Acc@1: 90.6250 (88.2890)  Acc@5: 100.0000 (99.7215)LR: 1.300e-02
Train: 25 [ 150/390]  Loss: 0.3777 (0.333)  Acc@1: 90.6250 (88.4209)  Acc@5: 100.0000 (99.6999)LR: 1.300e-02
Train: 25 [ 200/390]  Loss: 0.2145 (0.332)  Acc@1: 92.1875 (88.4328)  Acc@5: 100.0000 (99.6891)LR: 1.300e-02
Train: 25 [ 250/390]  Loss: 0.3159 (0.340)  Acc@1: 89.0625 (88.1412)  Acc@5: 100.0000 (99.6701)LR: 1.300e-02
Train: 25 [ 300/390]  Loss: 0.1732 (0.347)  Acc@1: 96.8750 (87.9101)  Acc@5: 100.0000 (99.6730)LR: 1.300e-02
Train: 25 [ 350/390]  Loss: 0.3202 (0.346)  Acc@1: 90.6250 (87.9763)  Acc@5: 100.0000 (99.6973)LR: 1.300e-02
Train: 25 [ 390/390]  Loss: 0.3538 (0.346)  Acc@1: 90.0000 (87.9600)  Acc@5: 100.0000 (99.6960)LR: 1.300e-02
train_acc 87.960000
Valid: 25 [   0/390]  Loss: 0.2803 (0.280)  Acc@1: 90.6250 (90.6250)  Acc@5: 100.0000 (100.0000)
Valid: 25 [  50/390]  Loss: 0.3518 (0.452)  Acc@1: 87.5000 (84.7426)  Acc@5: 100.0000 (99.2953)
Valid: 25 [ 100/390]  Loss: 0.4189 (0.471)  Acc@1: 85.9375 (84.1275)  Acc@5: 98.4375 (99.2729)
Valid: 25 [ 150/390]  Loss: 0.3018 (0.472)  Acc@1: 89.0625 (84.2508)  Acc@5: 100.0000 (99.2757)
Valid: 25 [ 200/390]  Loss: 0.3359 (0.475)  Acc@1: 92.1875 (84.0951)  Acc@5: 98.4375 (99.2615)
Valid: 25 [ 250/390]  Loss: 0.1859 (0.475)  Acc@1: 95.3125 (84.1322)  Acc@5: 100.0000 (99.2343)
Valid: 25 [ 300/390]  Loss: 0.5469 (0.477)  Acc@1: 81.2500 (83.9857)  Acc@5: 100.0000 (99.2681)
Valid: 25 [ 350/390]  Loss: 0.3965 (0.476)  Acc@1: 85.9375 (83.9966)  Acc@5: 100.0000 (99.2299)
Valid: 25 [ 390/390]  Loss: 0.6626 (0.479)  Acc@1: 77.5000 (83.8800)  Acc@5: 97.5000 (99.2280)
valid_acc 83.880000
epoch = 25   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3004, 0.6996],
        [0.2868, 0.7132],
        [0.2730, 0.7270],
        [0.2315, 0.7685],
        [0.3142, 0.6858],
        [0.3450, 0.6550],
        [0.2704, 0.7296],
        [0.4069, 0.5931],
        [0.4176, 0.5824],
        [0.3316, 0.6684],
        [0.3550, 0.6450],
        [0.4014, 0.5986],
        [0.3992, 0.6008],
        [0.3816, 0.6184]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.2422, 0.7578],
        [0.2578, 0.7422],
        [0.3023, 0.6977],
        [0.3200, 0.6800],
        [0.3694, 0.6306],
        [0.2385, 0.7615],
        [0.2772, 0.7228],
        [0.3524, 0.6476],
        [0.3771, 0.6229],
        [0.2872, 0.7128],
        [0.3085, 0.6915],
        [0.3626, 0.6374],
        [0.3890, 0.6110],
        [0.3851, 0.6149]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 26 [   0/390]  Loss: 0.3295 (0.330)  Acc@1: 87.5000 (87.5000)  Acc@5: 100.0000 (100.0000)LR: 1.225e-02
Train: 26 [  50/390]  Loss: 0.2617 (0.306)  Acc@1: 90.6250 (89.3995)  Acc@5: 100.0000 (99.8775)LR: 1.225e-02
Train: 26 [ 100/390]  Loss: 0.2992 (0.327)  Acc@1: 93.7500 (88.5829)  Acc@5: 100.0000 (99.7679)LR: 1.225e-02
Train: 26 [ 150/390]  Loss: 0.3938 (0.321)  Acc@1: 90.6250 (88.7728)  Acc@5: 100.0000 (99.7620)LR: 1.225e-02
Train: 26 [ 200/390]  Loss: 0.2631 (0.322)  Acc@1: 89.0625 (88.6971)  Acc@5: 100.0000 (99.7435)LR: 1.225e-02
Train: 26 [ 250/390]  Loss: 0.2627 (0.322)  Acc@1: 89.0625 (88.5645)  Acc@5: 100.0000 (99.7759)LR: 1.225e-02
Train: 26 [ 300/390]  Loss: 0.6163 (0.325)  Acc@1: 79.6875 (88.4603)  Acc@5: 100.0000 (99.7820)LR: 1.225e-02
Train: 26 [ 350/390]  Loss: 0.3471 (0.326)  Acc@1: 89.0625 (88.5105)  Acc@5: 98.4375 (99.7507)LR: 1.225e-02
Train: 26 [ 390/390]  Loss: 0.6392 (0.327)  Acc@1: 72.5000 (88.5080)  Acc@5: 100.0000 (99.7560)LR: 1.225e-02
train_acc 88.508000
Valid: 26 [   0/390]  Loss: 0.5542 (0.554)  Acc@1: 82.8125 (82.8125)  Acc@5: 98.4375 (98.4375)
Valid: 26 [  50/390]  Loss: 0.5562 (0.496)  Acc@1: 76.5625 (81.9240)  Acc@5: 98.4375 (99.1728)
Valid: 26 [ 100/390]  Loss: 0.5278 (0.503)  Acc@1: 81.2500 (82.3639)  Acc@5: 98.4375 (99.2110)
Valid: 26 [ 150/390]  Loss: 0.6377 (0.501)  Acc@1: 78.1250 (82.6159)  Acc@5: 96.8750 (99.1101)
Valid: 26 [ 200/390]  Loss: 0.5024 (0.505)  Acc@1: 85.9375 (82.5948)  Acc@5: 98.4375 (99.0749)
Valid: 26 [ 250/390]  Loss: 0.8979 (0.508)  Acc@1: 78.1250 (82.4639)  Acc@5: 93.7500 (99.0476)
Valid: 26 [ 300/390]  Loss: 0.4673 (0.504)  Acc@1: 81.2500 (82.6308)  Acc@5: 100.0000 (99.0760)
Valid: 26 [ 350/390]  Loss: 0.5088 (0.500)  Acc@1: 79.6875 (82.7190)  Acc@5: 100.0000 (99.0830)
Valid: 26 [ 390/390]  Loss: 0.2021 (0.500)  Acc@1: 95.0000 (82.7000)  Acc@5: 100.0000 (99.1080)
valid_acc 82.700000
epoch = 26   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3016, 0.6984],
        [0.2858, 0.7142],
        [0.2711, 0.7289],
        [0.2316, 0.7684],
        [0.3132, 0.6868],
        [0.3473, 0.6527],
        [0.2725, 0.7275],
        [0.4117, 0.5883],
        [0.4162, 0.5838],
        [0.3337, 0.6663],
        [0.3575, 0.6425],
        [0.4055, 0.5945],
        [0.3987, 0.6013],
        [0.3829, 0.6171]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.2385, 0.7615],
        [0.2609, 0.7391],
        [0.2945, 0.7055],
        [0.3177, 0.6823],
        [0.3704, 0.6296],
        [0.2265, 0.7735],
        [0.2714, 0.7286],
        [0.3549, 0.6451],
        [0.3752, 0.6248],
        [0.2807, 0.7193],
        [0.3102, 0.6898],
        [0.3635, 0.6365],
        [0.3865, 0.6135],
        [0.3904, 0.6096]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 27 [   0/390]  Loss: 0.3277 (0.328)  Acc@1: 90.6250 (90.6250)  Acc@5: 100.0000 (100.0000)LR: 1.150e-02
Train: 27 [  50/390]  Loss: 0.2176 (0.300)  Acc@1: 93.7500 (89.6752)  Acc@5: 100.0000 (99.8468)LR: 1.150e-02
Train: 27 [ 100/390]  Loss: 0.2809 (0.312)  Acc@1: 90.6250 (89.2791)  Acc@5: 100.0000 (99.7525)LR: 1.150e-02
Train: 27 [ 150/390]  Loss: 0.3488 (0.310)  Acc@1: 87.5000 (89.4040)  Acc@5: 100.0000 (99.7310)LR: 1.150e-02
Train: 27 [ 200/390]  Loss: 0.1529 (0.308)  Acc@1: 96.8750 (89.4434)  Acc@5: 100.0000 (99.6813)LR: 1.150e-02
Train: 27 [ 250/390]  Loss: 0.1923 (0.312)  Acc@1: 93.7500 (89.1621)  Acc@5: 100.0000 (99.7012)LR: 1.150e-02
Train: 27 [ 300/390]  Loss: 0.3130 (0.313)  Acc@1: 89.0625 (89.1300)  Acc@5: 100.0000 (99.7249)LR: 1.150e-02
Train: 27 [ 350/390]  Loss: 0.3729 (0.314)  Acc@1: 87.5000 (89.0803)  Acc@5: 98.4375 (99.7062)LR: 1.150e-02
Train: 27 [ 390/390]  Loss: 0.6013 (0.315)  Acc@1: 77.5000 (89.0840)  Acc@5: 97.5000 (99.7000)LR: 1.150e-02
train_acc 89.084000
Valid: 27 [   0/390]  Loss: 0.4209 (0.421)  Acc@1: 82.8125 (82.8125)  Acc@5: 100.0000 (100.0000)
Valid: 27 [  50/390]  Loss: 0.5015 (0.479)  Acc@1: 81.2500 (83.4559)  Acc@5: 98.4375 (98.9583)
Valid: 27 [ 100/390]  Loss: 0.3911 (0.485)  Acc@1: 87.5000 (83.5551)  Acc@5: 100.0000 (98.9790)
Valid: 27 [ 150/390]  Loss: 0.3350 (0.480)  Acc@1: 84.3750 (83.7231)  Acc@5: 100.0000 (99.0894)
Valid: 27 [ 200/390]  Loss: 0.6367 (0.473)  Acc@1: 79.6875 (83.8231)  Acc@5: 100.0000 (99.1604)
Valid: 27 [ 250/390]  Loss: 0.4705 (0.476)  Acc@1: 84.3750 (83.7463)  Acc@5: 100.0000 (99.1596)
Valid: 27 [ 300/390]  Loss: 0.6953 (0.482)  Acc@1: 82.8125 (83.6379)  Acc@5: 98.4375 (99.1539)
Valid: 27 [ 350/390]  Loss: 0.3362 (0.483)  Acc@1: 87.5000 (83.5737)  Acc@5: 98.4375 (99.1765)
Valid: 27 [ 390/390]  Loss: 0.2969 (0.483)  Acc@1: 90.0000 (83.5400)  Acc@5: 100.0000 (99.1840)
valid_acc 83.540000
epoch = 27   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3024, 0.6976],
        [0.2814, 0.7186],
        [0.2721, 0.7279],
        [0.2278, 0.7722],
        [0.3140, 0.6860],
        [0.3473, 0.6527],
        [0.2720, 0.7280],
        [0.4120, 0.5880],
        [0.4168, 0.5832],
        [0.3383, 0.6617],
        [0.3585, 0.6415],
        [0.4104, 0.5896],
        [0.4009, 0.5991],
        [0.3840, 0.6160]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.2350, 0.7650],
        [0.2584, 0.7416],
        [0.2893, 0.7107],
        [0.3097, 0.6903],
        [0.3692, 0.6308],
        [0.2200, 0.7800],
        [0.2704, 0.7296],
        [0.3529, 0.6471],
        [0.3694, 0.6306],
        [0.2750, 0.7250],
        [0.2991, 0.7009],
        [0.3627, 0.6373],
        [0.3833, 0.6167],
        [0.3934, 0.6066]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 28 [   0/390]  Loss: 0.2472 (0.247)  Acc@1: 92.1875 (92.1875)  Acc@5: 100.0000 (100.0000)LR: 1.075e-02
Train: 28 [  50/390]  Loss: 0.4344 (0.306)  Acc@1: 87.5000 (89.3689)  Acc@5: 100.0000 (99.8162)LR: 1.075e-02
Train: 28 [ 100/390]  Loss: 0.2328 (0.294)  Acc@1: 90.6250 (90.0990)  Acc@5: 100.0000 (99.8298)LR: 1.075e-02
Train: 28 [ 150/390]  Loss: 0.1326 (0.298)  Acc@1: 96.8750 (89.9110)  Acc@5: 100.0000 (99.8448)LR: 1.075e-02
Train: 28 [ 200/390]  Loss: 0.2061 (0.299)  Acc@1: 95.3125 (89.8321)  Acc@5: 100.0000 (99.8134)LR: 1.075e-02
Train: 28 [ 250/390]  Loss: 0.3091 (0.298)  Acc@1: 90.6250 (89.8469)  Acc@5: 100.0000 (99.8008)LR: 1.075e-02
Train: 28 [ 300/390]  Loss: 0.4557 (0.295)  Acc@1: 84.3750 (89.9502)  Acc@5: 100.0000 (99.7872)LR: 1.075e-02
Train: 28 [ 350/390]  Loss: 0.2530 (0.297)  Acc@1: 93.7500 (89.8682)  Acc@5: 100.0000 (99.7730)LR: 1.075e-02
Train: 28 [ 390/390]  Loss: 0.7163 (0.298)  Acc@1: 80.0000 (89.7760)  Acc@5: 100.0000 (99.7680)LR: 1.075e-02
train_acc 89.776000
Valid: 28 [   0/390]  Loss: 0.4768 (0.477)  Acc@1: 81.2500 (81.2500)  Acc@5: 100.0000 (100.0000)
Valid: 28 [  50/390]  Loss: 0.5884 (0.443)  Acc@1: 79.6875 (84.5588)  Acc@5: 100.0000 (99.3566)
Valid: 28 [ 100/390]  Loss: 0.2981 (0.439)  Acc@1: 90.6250 (85.2723)  Acc@5: 100.0000 (99.3502)
Valid: 28 [ 150/390]  Loss: 0.4827 (0.438)  Acc@1: 85.9375 (85.3994)  Acc@5: 98.4375 (99.4102)
Valid: 28 [ 200/390]  Loss: 0.4221 (0.439)  Acc@1: 85.9375 (85.1757)  Acc@5: 98.4375 (99.3859)
Valid: 28 [ 250/390]  Loss: 0.5820 (0.443)  Acc@1: 82.8125 (84.9602)  Acc@5: 98.4375 (99.3775)
Valid: 28 [ 300/390]  Loss: 0.4341 (0.444)  Acc@1: 87.5000 (84.8941)  Acc@5: 98.4375 (99.3978)
Valid: 28 [ 350/390]  Loss: 0.7739 (0.445)  Acc@1: 81.2500 (84.8513)  Acc@5: 96.8750 (99.3812)
Valid: 28 [ 390/390]  Loss: 0.5405 (0.445)  Acc@1: 87.5000 (84.8480)  Acc@5: 97.5000 (99.3800)
valid_acc 84.848000
epoch = 28   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3037, 0.6963],
        [0.2805, 0.7195],
        [0.2723, 0.7277],
        [0.2289, 0.7711],
        [0.3133, 0.6867],
        [0.3486, 0.6514],
        [0.2741, 0.7259],
        [0.4171, 0.5829],
        [0.4250, 0.5750],
        [0.3402, 0.6598],
        [0.3617, 0.6383],
        [0.4109, 0.5891],
        [0.4046, 0.5954],
        [0.3858, 0.6142]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.2318, 0.7682],
        [0.2554, 0.7446],
        [0.2866, 0.7134],
        [0.3072, 0.6928],
        [0.3648, 0.6352],
        [0.2179, 0.7821],
        [0.2642, 0.7358],
        [0.3501, 0.6499],
        [0.3641, 0.6359],
        [0.2737, 0.7263],
        [0.2991, 0.7009],
        [0.3581, 0.6419],
        [0.3786, 0.6214],
        [0.3939, 0.6061]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 29 [   0/390]  Loss: 0.3051 (0.305)  Acc@1: 89.0625 (89.0625)  Acc@5: 100.0000 (100.0000)LR: 1.002e-02
Train: 29 [  50/390]  Loss: 0.3502 (0.274)  Acc@1: 85.9375 (90.4718)  Acc@5: 100.0000 (99.7243)LR: 1.002e-02
Train: 29 [ 100/390]  Loss: 0.4030 (0.276)  Acc@1: 89.0625 (90.6250)  Acc@5: 100.0000 (99.7834)LR: 1.002e-02
Train: 29 [ 150/390]  Loss: 0.1685 (0.271)  Acc@1: 95.3125 (90.6043)  Acc@5: 100.0000 (99.8448)LR: 1.002e-02
Train: 29 [ 200/390]  Loss: 0.2969 (0.270)  Acc@1: 85.9375 (90.5395)  Acc@5: 100.0000 (99.8523)LR: 1.002e-02
Train: 29 [ 250/390]  Loss: 0.3182 (0.271)  Acc@1: 85.9375 (90.4258)  Acc@5: 100.0000 (99.8381)LR: 1.002e-02
Train: 29 [ 300/390]  Loss: 0.1595 (0.277)  Acc@1: 95.3125 (90.2876)  Acc@5: 100.0000 (99.8391)LR: 1.002e-02
Train: 29 [ 350/390]  Loss: 0.2854 (0.281)  Acc@1: 92.1875 (90.2288)  Acc@5: 100.0000 (99.8397)LR: 1.002e-02
Train: 29 [ 390/390]  Loss: 0.2992 (0.281)  Acc@1: 87.5000 (90.1960)  Acc@5: 100.0000 (99.8440)LR: 1.002e-02
train_acc 90.196000
Valid: 29 [   0/390]  Loss: 0.7427 (0.743)  Acc@1: 76.5625 (76.5625)  Acc@5: 100.0000 (100.0000)
Valid: 29 [  50/390]  Loss: 0.3525 (0.424)  Acc@1: 82.8125 (85.1716)  Acc@5: 100.0000 (99.6324)
Valid: 29 [ 100/390]  Loss: 0.2861 (0.433)  Acc@1: 82.8125 (84.8855)  Acc@5: 100.0000 (99.4740)
Valid: 29 [ 150/390]  Loss: 0.4858 (0.426)  Acc@1: 79.6875 (85.2649)  Acc@5: 100.0000 (99.4723)
Valid: 29 [ 200/390]  Loss: 0.4863 (0.430)  Acc@1: 85.9375 (85.1524)  Acc@5: 98.4375 (99.4481)
Valid: 29 [ 250/390]  Loss: 0.3384 (0.434)  Acc@1: 85.9375 (85.0722)  Acc@5: 100.0000 (99.4086)
Valid: 29 [ 300/390]  Loss: 0.5811 (0.438)  Acc@1: 81.2500 (84.9772)  Acc@5: 98.4375 (99.3771)
Valid: 29 [ 350/390]  Loss: 0.4883 (0.438)  Acc@1: 79.6875 (85.0249)  Acc@5: 100.0000 (99.3456)
Valid: 29 [ 390/390]  Loss: 0.2937 (0.438)  Acc@1: 92.5000 (85.0040)  Acc@5: 100.0000 (99.3560)
valid_acc 85.004000
epoch = 29   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3083, 0.6917],
        [0.2790, 0.7210],
        [0.2752, 0.7248],
        [0.2297, 0.7703],
        [0.3133, 0.6867],
        [0.3532, 0.6468],
        [0.2738, 0.7262],
        [0.4214, 0.5786],
        [0.4314, 0.5686],
        [0.3434, 0.6566],
        [0.3656, 0.6344],
        [0.4065, 0.5935],
        [0.4082, 0.5918],
        [0.3848, 0.6152]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.2311, 0.7689],
        [0.2515, 0.7485],
        [0.2857, 0.7143],
        [0.3033, 0.6967],
        [0.3610, 0.6390],
        [0.2146, 0.7854],
        [0.2611, 0.7389],
        [0.3486, 0.6514],
        [0.3639, 0.6361],
        [0.2695, 0.7305],
        [0.2960, 0.7040],
        [0.3561, 0.6439],
        [0.3795, 0.6205],
        [0.3926, 0.6074]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 30 [   0/390]  Loss: 0.2366 (0.237)  Acc@1: 92.1875 (92.1875)  Acc@5: 100.0000 (100.0000)LR: 9.292e-03
Train: 30 [  50/390]  Loss: 0.1990 (0.267)  Acc@1: 92.1875 (90.7475)  Acc@5: 100.0000 (99.8775)LR: 9.292e-03
Train: 30 [ 100/390]  Loss: 0.2928 (0.257)  Acc@1: 85.9375 (90.8880)  Acc@5: 100.0000 (99.9072)LR: 9.292e-03
Train: 30 [ 150/390]  Loss: 0.1797 (0.259)  Acc@1: 93.7500 (90.9768)  Acc@5: 100.0000 (99.8655)LR: 9.292e-03
Train: 30 [ 200/390]  Loss: 0.3238 (0.262)  Acc@1: 87.5000 (90.7960)  Acc@5: 100.0000 (99.8057)LR: 9.292e-03
Train: 30 [ 250/390]  Loss: 0.3820 (0.270)  Acc@1: 90.6250 (90.5939)  Acc@5: 100.0000 (99.7883)LR: 9.292e-03
Train: 30 [ 300/390]  Loss: 0.3284 (0.271)  Acc@1: 87.5000 (90.6302)  Acc@5: 100.0000 (99.7820)LR: 9.292e-03
Train: 30 [ 350/390]  Loss: 0.3381 (0.269)  Acc@1: 89.0625 (90.5983)  Acc@5: 100.0000 (99.7819)LR: 9.292e-03
Train: 30 [ 390/390]  Loss: 0.1530 (0.270)  Acc@1: 92.5000 (90.5480)  Acc@5: 100.0000 (99.7920)LR: 9.292e-03
train_acc 90.548000
Valid: 30 [   0/390]  Loss: 0.2666 (0.267)  Acc@1: 89.0625 (89.0625)  Acc@5: 100.0000 (100.0000)
Valid: 30 [  50/390]  Loss: 0.3303 (0.435)  Acc@1: 89.0625 (84.9877)  Acc@5: 100.0000 (99.2647)
Valid: 30 [ 100/390]  Loss: 0.3162 (0.434)  Acc@1: 87.5000 (85.2723)  Acc@5: 100.0000 (99.3193)
Valid: 30 [ 150/390]  Loss: 0.3916 (0.438)  Acc@1: 90.6250 (85.1200)  Acc@5: 100.0000 (99.3067)
Valid: 30 [ 200/390]  Loss: 0.5439 (0.431)  Acc@1: 82.8125 (85.3700)  Acc@5: 100.0000 (99.3470)
Valid: 30 [ 250/390]  Loss: 0.4497 (0.429)  Acc@1: 85.9375 (85.4021)  Acc@5: 98.4375 (99.3775)
Valid: 30 [ 300/390]  Loss: 0.3669 (0.427)  Acc@1: 87.5000 (85.5430)  Acc@5: 100.0000 (99.4186)
Valid: 30 [ 350/390]  Loss: 0.4902 (0.429)  Acc@1: 89.0625 (85.5413)  Acc@5: 98.4375 (99.3990)
Valid: 30 [ 390/390]  Loss: 0.1965 (0.426)  Acc@1: 97.5000 (85.6240)  Acc@5: 100.0000 (99.4120)
valid_acc 85.624000
epoch = 30   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3120, 0.6880],
        [0.2788, 0.7212],
        [0.2807, 0.7193],
        [0.2307, 0.7693],
        [0.3169, 0.6831],
        [0.3590, 0.6410],
        [0.2793, 0.7207],
        [0.4239, 0.5761],
        [0.4307, 0.5693],
        [0.3453, 0.6547],
        [0.3665, 0.6335],
        [0.4061, 0.5939],
        [0.4114, 0.5886],
        [0.3841, 0.6159]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.2300, 0.7700],
        [0.2470, 0.7530],
        [0.2803, 0.7197],
        [0.2991, 0.7009],
        [0.3612, 0.6388],
        [0.2109, 0.7891],
        [0.2580, 0.7420],
        [0.3494, 0.6506],
        [0.3586, 0.6414],
        [0.2666, 0.7334],
        [0.2943, 0.7057],
        [0.3511, 0.6489],
        [0.3779, 0.6221],
        [0.3944, 0.6056]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 31 [   0/390]  Loss: 0.2744 (0.274)  Acc@1: 95.3125 (95.3125)  Acc@5: 98.4375 (98.4375)LR: 8.583e-03
Train: 31 [  50/390]  Loss: 0.1681 (0.242)  Acc@1: 98.4375 (91.8199)  Acc@5: 100.0000 (99.9081)LR: 8.583e-03
Train: 31 [ 100/390]  Loss: 0.3093 (0.259)  Acc@1: 89.0625 (91.2438)  Acc@5: 98.4375 (99.8298)LR: 8.583e-03
Train: 31 [ 150/390]  Loss: 0.3695 (0.261)  Acc@1: 87.5000 (91.1010)  Acc@5: 100.0000 (99.8241)LR: 8.583e-03
Train: 31 [ 200/390]  Loss: 0.2447 (0.256)  Acc@1: 92.1875 (91.1925)  Acc@5: 100.0000 (99.8368)LR: 8.583e-03
Train: 31 [ 250/390]  Loss: 0.2639 (0.256)  Acc@1: 93.7500 (91.2164)  Acc@5: 100.0000 (99.8381)LR: 8.583e-03
Train: 31 [ 300/390]  Loss: 0.3559 (0.256)  Acc@1: 85.9375 (91.2116)  Acc@5: 98.4375 (99.8287)LR: 8.583e-03
Train: 31 [ 350/390]  Loss: 0.2448 (0.256)  Acc@1: 93.7500 (91.1993)  Acc@5: 100.0000 (99.8264)LR: 8.583e-03
Train: 31 [ 390/390]  Loss: 0.3122 (0.255)  Acc@1: 87.5000 (91.2080)  Acc@5: 100.0000 (99.8240)LR: 8.583e-03
train_acc 91.208000
Valid: 31 [   0/390]  Loss: 0.3772 (0.377)  Acc@1: 92.1875 (92.1875)  Acc@5: 100.0000 (100.0000)
Valid: 31 [  50/390]  Loss: 0.3254 (0.434)  Acc@1: 89.0625 (84.9571)  Acc@5: 100.0000 (99.3260)
Valid: 31 [ 100/390]  Loss: 0.2971 (0.451)  Acc@1: 89.0625 (84.7772)  Acc@5: 100.0000 (99.4121)
Valid: 31 [ 150/390]  Loss: 0.7061 (0.447)  Acc@1: 78.1250 (85.0890)  Acc@5: 98.4375 (99.4412)
Valid: 31 [ 200/390]  Loss: 0.5000 (0.448)  Acc@1: 82.8125 (85.0513)  Acc@5: 100.0000 (99.4170)
Valid: 31 [ 250/390]  Loss: 0.3418 (0.448)  Acc@1: 85.9375 (84.9913)  Acc@5: 100.0000 (99.3650)
Valid: 31 [ 300/390]  Loss: 0.4060 (0.448)  Acc@1: 85.9375 (85.1537)  Acc@5: 98.4375 (99.3355)
Valid: 31 [ 350/390]  Loss: 0.2502 (0.454)  Acc@1: 95.3125 (85.0160)  Acc@5: 100.0000 (99.3234)
Valid: 31 [ 390/390]  Loss: 0.4805 (0.449)  Acc@1: 82.5000 (85.1360)  Acc@5: 100.0000 (99.3240)
valid_acc 85.136000
epoch = 31   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3122, 0.6878],
        [0.2828, 0.7172],
        [0.2804, 0.7196],
        [0.2323, 0.7677],
        [0.3181, 0.6819],
        [0.3693, 0.6307],
        [0.2860, 0.7140],
        [0.4244, 0.5756],
        [0.4329, 0.5671],
        [0.3467, 0.6533],
        [0.3725, 0.6275],
        [0.4112, 0.5888],
        [0.4140, 0.5860],
        [0.3887, 0.6113]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.2274, 0.7726],
        [0.2451, 0.7549],
        [0.2767, 0.7233],
        [0.2964, 0.7036],
        [0.3600, 0.6400],
        [0.2075, 0.7925],
        [0.2546, 0.7454],
        [0.3494, 0.6506],
        [0.3517, 0.6483],
        [0.2625, 0.7375],
        [0.2935, 0.7065],
        [0.3533, 0.6467],
        [0.3729, 0.6271],
        [0.3929, 0.6071]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 32 [   0/390]  Loss: 0.3405 (0.340)  Acc@1: 90.6250 (90.6250)  Acc@5: 98.4375 (98.4375)LR: 7.891e-03
Train: 32 [  50/390]  Loss: 0.2679 (0.249)  Acc@1: 92.1875 (91.4522)  Acc@5: 100.0000 (99.8162)LR: 7.891e-03
Train: 32 [ 100/390]  Loss: 0.2125 (0.237)  Acc@1: 95.3125 (91.8781)  Acc@5: 100.0000 (99.9072)LR: 7.891e-03
Train: 32 [ 150/390]  Loss: 0.2815 (0.240)  Acc@1: 89.0625 (91.7943)  Acc@5: 100.0000 (99.8965)LR: 7.891e-03
Train: 32 [ 200/390]  Loss: 0.1684 (0.239)  Acc@1: 95.3125 (91.8066)  Acc@5: 100.0000 (99.8989)LR: 7.891e-03
Train: 32 [ 250/390]  Loss: 0.3429 (0.239)  Acc@1: 87.5000 (91.7019)  Acc@5: 100.0000 (99.9128)LR: 7.891e-03
Train: 32 [ 300/390]  Loss: 0.3076 (0.239)  Acc@1: 87.5000 (91.6840)  Acc@5: 100.0000 (99.8910)LR: 7.891e-03
Train: 32 [ 350/390]  Loss: 0.1836 (0.238)  Acc@1: 92.1875 (91.7379)  Acc@5: 100.0000 (99.9021)LR: 7.891e-03
Train: 32 [ 390/390]  Loss: 0.5231 (0.243)  Acc@1: 80.0000 (91.5440)  Acc@5: 100.0000 (99.8920)LR: 7.891e-03
train_acc 91.544000
Valid: 32 [   0/390]  Loss: 0.3284 (0.328)  Acc@1: 85.9375 (85.9375)  Acc@5: 100.0000 (100.0000)
Valid: 32 [  50/390]  Loss: 0.3628 (0.410)  Acc@1: 84.3750 (86.1520)  Acc@5: 98.4375 (99.5098)
Valid: 32 [ 100/390]  Loss: 0.3975 (0.412)  Acc@1: 85.9375 (85.9684)  Acc@5: 98.4375 (99.5359)
Valid: 32 [ 150/390]  Loss: 0.5552 (0.416)  Acc@1: 78.1250 (85.9996)  Acc@5: 98.4375 (99.4516)
Valid: 32 [ 200/390]  Loss: 0.4148 (0.420)  Acc@1: 84.3750 (85.9142)  Acc@5: 100.0000 (99.4092)
Valid: 32 [ 250/390]  Loss: 0.3152 (0.423)  Acc@1: 87.5000 (86.0060)  Acc@5: 100.0000 (99.3775)
Valid: 32 [ 300/390]  Loss: 0.5884 (0.427)  Acc@1: 82.8125 (85.8804)  Acc@5: 100.0000 (99.3926)
Valid: 32 [ 350/390]  Loss: 0.4539 (0.428)  Acc@1: 87.5000 (85.7772)  Acc@5: 100.0000 (99.3857)
Valid: 32 [ 390/390]  Loss: 0.9316 (0.430)  Acc@1: 75.0000 (85.6840)  Acc@5: 100.0000 (99.3920)
valid_acc 85.684000
epoch = 32   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3140, 0.6860],
        [0.2870, 0.7130],
        [0.2821, 0.7179],
        [0.2323, 0.7677],
        [0.3232, 0.6768],
        [0.3764, 0.6236],
        [0.2931, 0.7069],
        [0.4293, 0.5707],
        [0.4342, 0.5658],
        [0.3541, 0.6459],
        [0.3790, 0.6210],
        [0.4159, 0.5841],
        [0.4120, 0.5880],
        [0.3923, 0.6077]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.2296, 0.7704],
        [0.2400, 0.7600],
        [0.2775, 0.7225],
        [0.2950, 0.7050],
        [0.3550, 0.6450],
        [0.2030, 0.7970],
        [0.2530, 0.7470],
        [0.3479, 0.6521],
        [0.3469, 0.6531],
        [0.2575, 0.7425],
        [0.2924, 0.7076],
        [0.3498, 0.6502],
        [0.3688, 0.6312],
        [0.3929, 0.6071]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 33 [   0/390]  Loss: 0.1812 (0.181)  Acc@1: 92.1875 (92.1875)  Acc@5: 100.0000 (100.0000)LR: 7.219e-03
Train: 33 [  50/390]  Loss: 0.2958 (0.218)  Acc@1: 90.6250 (92.4326)  Acc@5: 100.0000 (99.8468)LR: 7.219e-03
Train: 33 [ 100/390]  Loss: 0.1969 (0.224)  Acc@1: 93.7500 (92.1875)  Acc@5: 100.0000 (99.8762)LR: 7.219e-03
Train: 33 [ 150/390]  Loss: 0.3324 (0.224)  Acc@1: 89.0625 (92.1668)  Acc@5: 100.0000 (99.8655)LR: 7.219e-03
Train: 33 [ 200/390]  Loss: 0.1891 (0.221)  Acc@1: 92.1875 (92.1953)  Acc@5: 100.0000 (99.8756)LR: 7.219e-03
Train: 33 [ 250/390]  Loss: 0.1390 (0.219)  Acc@1: 96.8750 (92.3307)  Acc@5: 100.0000 (99.8942)LR: 7.219e-03
Train: 33 [ 300/390]  Loss: 0.08061 (0.223)  Acc@1: 98.4375 (92.2342)  Acc@5: 100.0000 (99.8910)LR: 7.219e-03
Train: 33 [ 350/390]  Loss: 0.2789 (0.225)  Acc@1: 92.1875 (92.1563)  Acc@5: 100.0000 (99.9021)LR: 7.219e-03
Train: 33 [ 390/390]  Loss: 0.2154 (0.227)  Acc@1: 92.5000 (92.0320)  Acc@5: 100.0000 (99.9040)LR: 7.219e-03
train_acc 92.032000
Valid: 33 [   0/390]  Loss: 0.6587 (0.659)  Acc@1: 79.6875 (79.6875)  Acc@5: 98.4375 (98.4375)
Valid: 33 [  50/390]  Loss: 0.3821 (0.478)  Acc@1: 87.5000 (84.1605)  Acc@5: 100.0000 (99.2034)
Valid: 33 [ 100/390]  Loss: 0.4326 (0.455)  Acc@1: 85.9375 (84.9474)  Acc@5: 100.0000 (99.3657)
Valid: 33 [ 150/390]  Loss: 0.6094 (0.453)  Acc@1: 82.8125 (84.9338)  Acc@5: 98.4375 (99.3791)
Valid: 33 [ 200/390]  Loss: 0.2795 (0.460)  Acc@1: 90.6250 (84.7170)  Acc@5: 100.0000 (99.3781)
Valid: 33 [ 250/390]  Loss: 0.2915 (0.456)  Acc@1: 85.9375 (84.6987)  Acc@5: 100.0000 (99.3837)
Valid: 33 [ 300/390]  Loss: 0.4438 (0.456)  Acc@1: 82.8125 (84.6449)  Acc@5: 98.4375 (99.3823)
Valid: 33 [ 350/390]  Loss: 0.2854 (0.455)  Acc@1: 90.6250 (84.6955)  Acc@5: 98.4375 (99.3768)
Valid: 33 [ 390/390]  Loss: 0.9019 (0.455)  Acc@1: 75.0000 (84.6680)  Acc@5: 95.0000 (99.3640)
valid_acc 84.668000
epoch = 33   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3150, 0.6850],
        [0.2924, 0.7076],
        [0.2884, 0.7116],
        [0.2372, 0.7628],
        [0.3283, 0.6717],
        [0.3777, 0.6223],
        [0.3016, 0.6984],
        [0.4336, 0.5664],
        [0.4375, 0.5625],
        [0.3566, 0.6434],
        [0.3828, 0.6172],
        [0.4201, 0.5799],
        [0.4119, 0.5881],
        [0.3908, 0.6092]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.2289, 0.7711],
        [0.2361, 0.7639],
        [0.2749, 0.7251],
        [0.2879, 0.7121],
        [0.3553, 0.6447],
        [0.2004, 0.7996],
        [0.2492, 0.7508],
        [0.3483, 0.6517],
        [0.3477, 0.6523],
        [0.2555, 0.7445],
        [0.2900, 0.7100],
        [0.3436, 0.6564],
        [0.3647, 0.6353],
        [0.3925, 0.6075]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 34 [   0/390]  Loss: 0.2219 (0.222)  Acc@1: 90.6250 (90.6250)  Acc@5: 100.0000 (100.0000)LR: 6.570e-03
Train: 34 [  50/390]  Loss: 0.1641 (0.216)  Acc@1: 92.1875 (92.8922)  Acc@5: 100.0000 (99.8775)LR: 6.570e-03
Train: 34 [ 100/390]  Loss: 0.2739 (0.206)  Acc@1: 89.0625 (92.9301)  Acc@5: 100.0000 (99.9072)LR: 6.570e-03
Train: 34 [ 150/390]  Loss: 0.2644 (0.213)  Acc@1: 89.0625 (92.7256)  Acc@5: 100.0000 (99.8758)LR: 6.570e-03
Train: 34 [ 200/390]  Loss: 0.2089 (0.210)  Acc@1: 89.0625 (92.7705)  Acc@5: 100.0000 (99.8912)LR: 6.570e-03
Train: 34 [ 250/390]  Loss: 0.09475 (0.210)  Acc@1: 98.4375 (92.6170)  Acc@5: 100.0000 (99.8942)LR: 6.570e-03
Train: 34 [ 300/390]  Loss: 0.2287 (0.214)  Acc@1: 92.1875 (92.5768)  Acc@5: 100.0000 (99.8702)LR: 6.570e-03
Train: 34 [ 350/390]  Loss: 0.09960 (0.216)  Acc@1: 96.8750 (92.4769)  Acc@5: 100.0000 (99.8665)LR: 6.570e-03
Train: 34 [ 390/390]  Loss: 0.4499 (0.217)  Acc@1: 90.0000 (92.4600)  Acc@5: 97.5000 (99.8520)LR: 6.570e-03
train_acc 92.460000
Valid: 34 [   0/390]  Loss: 0.3342 (0.334)  Acc@1: 87.5000 (87.5000)  Acc@5: 98.4375 (98.4375)
Valid: 34 [  50/390]  Loss: 0.4697 (0.432)  Acc@1: 84.3750 (85.2635)  Acc@5: 100.0000 (99.4179)
Valid: 34 [ 100/390]  Loss: 0.3162 (0.446)  Acc@1: 92.1875 (85.1021)  Acc@5: 98.4375 (99.3812)
Valid: 34 [ 150/390]  Loss: 0.4011 (0.449)  Acc@1: 89.0625 (84.9027)  Acc@5: 100.0000 (99.4102)
Valid: 34 [ 200/390]  Loss: 0.4595 (0.452)  Acc@1: 82.8125 (84.8958)  Acc@5: 100.0000 (99.4170)
Valid: 34 [ 250/390]  Loss: 0.3889 (0.453)  Acc@1: 85.9375 (84.9477)  Acc@5: 100.0000 (99.4211)
Valid: 34 [ 300/390]  Loss: 0.5659 (0.454)  Acc@1: 79.6875 (84.8837)  Acc@5: 96.8750 (99.3926)
Valid: 34 [ 350/390]  Loss: 0.6123 (0.450)  Acc@1: 84.3750 (85.0249)  Acc@5: 100.0000 (99.4302)
Valid: 34 [ 390/390]  Loss: 0.3247 (0.451)  Acc@1: 85.0000 (84.9880)  Acc@5: 100.0000 (99.4360)
valid_acc 84.988000
epoch = 34   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 4)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3156, 0.6844],
        [0.2961, 0.7039],
        [0.2909, 0.7091],
        [0.2425, 0.7575],
        [0.3345, 0.6655],
        [0.3817, 0.6183],
        [0.3060, 0.6940],
        [0.4402, 0.5598],
        [0.4403, 0.5597],
        [0.3600, 0.6400],
        [0.3937, 0.6063],
        [0.4233, 0.5767],
        [0.4156, 0.5844],
        [0.3926, 0.6074]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.2264, 0.7736],
        [0.2350, 0.7650],
        [0.2708, 0.7292],
        [0.2810, 0.7190],
        [0.3526, 0.6474],
        [0.1979, 0.8021],
        [0.2453, 0.7547],
        [0.3497, 0.6503],
        [0.3429, 0.6571],
        [0.2498, 0.7502],
        [0.2929, 0.7071],
        [0.3441, 0.6559],
        [0.3634, 0.6366],
        [0.3936, 0.6064]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 35 [   0/390]  Loss: 0.1546 (0.155)  Acc@1: 95.3125 (95.3125)  Acc@5: 100.0000 (100.0000)LR: 5.947e-03
Train: 35 [  50/390]  Loss: 0.2934 (0.199)  Acc@1: 87.5000 (92.8615)  Acc@5: 100.0000 (99.9694)LR: 5.947e-03
Train: 35 [ 100/390]  Loss: 0.1867 (0.203)  Acc@1: 93.7500 (92.7908)  Acc@5: 100.0000 (99.9072)LR: 5.947e-03
Train: 35 [ 150/390]  Loss: 0.1478 (0.200)  Acc@1: 93.7500 (92.8084)  Acc@5: 100.0000 (99.8965)LR: 5.947e-03
Train: 35 [ 200/390]  Loss: 0.3488 (0.196)  Acc@1: 87.5000 (92.9571)  Acc@5: 98.4375 (99.8912)LR: 5.947e-03
Train: 35 [ 250/390]  Loss: 0.1482 (0.198)  Acc@1: 95.3125 (92.8660)  Acc@5: 100.0000 (99.8879)LR: 5.947e-03
Train: 35 [ 300/390]  Loss: 0.2469 (0.201)  Acc@1: 90.6250 (92.7274)  Acc@5: 100.0000 (99.8910)LR: 5.947e-03
Train: 35 [ 350/390]  Loss: 0.1308 (0.207)  Acc@1: 93.7500 (92.5080)  Acc@5: 100.0000 (99.8754)LR: 5.947e-03
Train: 35 [ 390/390]  Loss: 0.1237 (0.209)  Acc@1: 95.0000 (92.4520)  Acc@5: 100.0000 (99.8720)LR: 5.947e-03
train_acc 92.452000
Valid: 35 [   0/390]  Loss: 0.5098 (0.510)  Acc@1: 81.2500 (81.2500)  Acc@5: 98.4375 (98.4375)
Valid: 35 [  50/390]  Loss: 0.6367 (0.467)  Acc@1: 75.0000 (84.3444)  Acc@5: 98.4375 (99.3566)
Valid: 35 [ 100/390]  Loss: 0.2247 (0.465)  Acc@1: 87.5000 (84.4214)  Acc@5: 100.0000 (99.4121)
Valid: 35 [ 150/390]  Loss: 0.2834 (0.448)  Acc@1: 92.1875 (84.9959)  Acc@5: 98.4375 (99.3895)
Valid: 35 [ 200/390]  Loss: 0.8359 (0.446)  Acc@1: 76.5625 (84.9736)  Acc@5: 98.4375 (99.3937)
Valid: 35 [ 250/390]  Loss: 0.4451 (0.437)  Acc@1: 84.3750 (85.2403)  Acc@5: 100.0000 (99.4335)
Valid: 35 [ 300/390]  Loss: 0.5679 (0.438)  Acc@1: 82.8125 (85.2211)  Acc@5: 98.4375 (99.4290)
Valid: 35 [ 350/390]  Loss: 0.2656 (0.437)  Acc@1: 92.1875 (85.2609)  Acc@5: 100.0000 (99.4035)
Valid: 35 [ 390/390]  Loss: 0.4082 (0.436)  Acc@1: 87.5000 (85.3120)  Acc@5: 97.5000 (99.4040)
valid_acc 85.312000
epoch = 35   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 4)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3142, 0.6858],
        [0.2996, 0.7004],
        [0.2970, 0.7030],
        [0.2486, 0.7514],
        [0.3338, 0.6662],
        [0.3854, 0.6146],
        [0.3037, 0.6963],
        [0.4418, 0.5582],
        [0.4435, 0.5565],
        [0.3675, 0.6325],
        [0.4002, 0.5998],
        [0.4299, 0.5701],
        [0.4175, 0.5825],
        [0.3949, 0.6051]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.2234, 0.7766],
        [0.2340, 0.7660],
        [0.2673, 0.7327],
        [0.2774, 0.7226],
        [0.3539, 0.6461],
        [0.1912, 0.8088],
        [0.2481, 0.7519],
        [0.3530, 0.6470],
        [0.3440, 0.6560],
        [0.2428, 0.7572],
        [0.2941, 0.7059],
        [0.3480, 0.6520],
        [0.3631, 0.6369],
        [0.3985, 0.6015]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 36 [   0/390]  Loss: 0.08672 (0.0867)  Acc@1: 96.8750 (96.8750)  Acc@5: 100.0000 (100.0000)LR: 5.351e-03
Train: 36 [  50/390]  Loss: 0.2012 (0.202)  Acc@1: 92.1875 (93.2292)  Acc@5: 100.0000 (99.8468)LR: 5.351e-03
Train: 36 [ 100/390]  Loss: 0.3198 (0.194)  Acc@1: 87.5000 (93.3014)  Acc@5: 100.0000 (99.8917)LR: 5.351e-03
Train: 36 [ 150/390]  Loss: 0.1445 (0.191)  Acc@1: 93.7500 (93.2430)  Acc@5: 100.0000 (99.8758)LR: 5.351e-03
Train: 36 [ 200/390]  Loss: 0.2985 (0.191)  Acc@1: 84.3750 (93.1825)  Acc@5: 100.0000 (99.8834)LR: 5.351e-03
Train: 36 [ 250/390]  Loss: 0.2569 (0.195)  Acc@1: 93.7500 (93.1399)  Acc@5: 100.0000 (99.8630)LR: 5.351e-03
Train: 36 [ 300/390]  Loss: 0.2183 (0.193)  Acc@1: 96.8750 (93.2205)  Acc@5: 98.4375 (99.8702)LR: 5.351e-03
Train: 36 [ 350/390]  Loss: 0.2639 (0.194)  Acc@1: 95.3125 (93.1357)  Acc@5: 100.0000 (99.8843)LR: 5.351e-03
Train: 36 [ 390/390]  Loss: 0.1348 (0.193)  Acc@1: 92.5000 (93.1480)  Acc@5: 100.0000 (99.8920)LR: 5.351e-03
train_acc 93.148000
Valid: 36 [   0/390]  Loss: 0.2285 (0.229)  Acc@1: 92.1875 (92.1875)  Acc@5: 100.0000 (100.0000)
Valid: 36 [  50/390]  Loss: 0.6235 (0.401)  Acc@1: 81.2500 (86.6728)  Acc@5: 98.4375 (99.5711)
Valid: 36 [ 100/390]  Loss: 0.4900 (0.401)  Acc@1: 81.2500 (86.5254)  Acc@5: 100.0000 (99.5359)
Valid: 36 [ 150/390]  Loss: 0.4739 (0.406)  Acc@1: 89.0625 (86.5273)  Acc@5: 96.8750 (99.5033)
Valid: 36 [ 200/390]  Loss: 0.6260 (0.404)  Acc@1: 79.6875 (86.5361)  Acc@5: 100.0000 (99.4947)
Valid: 36 [ 250/390]  Loss: 0.1805 (0.403)  Acc@1: 90.6250 (86.5974)  Acc@5: 100.0000 (99.5082)
Valid: 36 [ 300/390]  Loss: 0.4292 (0.404)  Acc@1: 87.5000 (86.5500)  Acc@5: 100.0000 (99.4913)
Valid: 36 [ 350/390]  Loss: 0.5317 (0.404)  Acc@1: 82.8125 (86.5830)  Acc@5: 98.4375 (99.4836)
Valid: 36 [ 390/390]  Loss: 0.2294 (0.402)  Acc@1: 92.5000 (86.5920)  Acc@5: 100.0000 (99.4920)
valid_acc 86.592000
epoch = 36   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 4)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3141, 0.6859],
        [0.3033, 0.6967],
        [0.3009, 0.6991],
        [0.2528, 0.7472],
        [0.3387, 0.6613],
        [0.3921, 0.6079],
        [0.3079, 0.6921],
        [0.4527, 0.5473],
        [0.4457, 0.5543],
        [0.3726, 0.6274],
        [0.4038, 0.5962],
        [0.4313, 0.5687],
        [0.4224, 0.5776],
        [0.4027, 0.5973]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.2215, 0.7785],
        [0.2360, 0.7640],
        [0.2633, 0.7367],
        [0.2750, 0.7250],
        [0.3504, 0.6496],
        [0.1875, 0.8125],
        [0.2457, 0.7543],
        [0.3535, 0.6465],
        [0.3403, 0.6597],
        [0.2407, 0.7593],
        [0.2906, 0.7094],
        [0.3451, 0.6549],
        [0.3605, 0.6395],
        [0.4025, 0.5975]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 37 [   0/390]  Loss: 0.1362 (0.136)  Acc@1: 93.7500 (93.7500)  Acc@5: 100.0000 (100.0000)LR: 4.785e-03
Train: 37 [  50/390]  Loss: 0.1786 (0.195)  Acc@1: 92.1875 (92.7390)  Acc@5: 100.0000 (99.9387)LR: 4.785e-03
Train: 37 [ 100/390]  Loss: 0.1246 (0.186)  Acc@1: 95.3125 (93.0848)  Acc@5: 100.0000 (99.9072)LR: 4.785e-03
Train: 37 [ 150/390]  Loss: 0.1765 (0.185)  Acc@1: 93.7500 (93.3050)  Acc@5: 100.0000 (99.9172)LR: 4.785e-03
Train: 37 [ 200/390]  Loss: 0.2117 (0.188)  Acc@1: 90.6250 (93.2447)  Acc@5: 100.0000 (99.9145)LR: 4.785e-03
Train: 37 [ 250/390]  Loss: 0.1512 (0.185)  Acc@1: 93.7500 (93.4636)  Acc@5: 100.0000 (99.9253)LR: 4.785e-03
Train: 37 [ 300/390]  Loss: 0.1788 (0.183)  Acc@1: 95.3125 (93.4437)  Acc@5: 100.0000 (99.9066)LR: 4.785e-03
Train: 37 [ 350/390]  Loss: 0.2603 (0.184)  Acc@1: 89.0625 (93.3672)  Acc@5: 98.4375 (99.9021)LR: 4.785e-03
Train: 37 [ 390/390]  Loss: 0.1869 (0.184)  Acc@1: 90.0000 (93.3480)  Acc@5: 100.0000 (99.9040)LR: 4.785e-03
train_acc 93.348000
Valid: 37 [   0/390]  Loss: 0.5410 (0.541)  Acc@1: 81.2500 (81.2500)  Acc@5: 100.0000 (100.0000)
Valid: 37 [  50/390]  Loss: 0.6377 (0.434)  Acc@1: 85.9375 (85.9988)  Acc@5: 98.4375 (99.3566)
Valid: 37 [ 100/390]  Loss: 0.3337 (0.410)  Acc@1: 85.9375 (86.5254)  Acc@5: 100.0000 (99.4121)
Valid: 37 [ 150/390]  Loss: 0.2462 (0.401)  Acc@1: 90.6250 (86.7757)  Acc@5: 100.0000 (99.3791)
Valid: 37 [ 200/390]  Loss: 0.2332 (0.408)  Acc@1: 92.1875 (86.5438)  Acc@5: 100.0000 (99.4248)
Valid: 37 [ 250/390]  Loss: 0.3054 (0.405)  Acc@1: 89.0625 (86.6409)  Acc@5: 100.0000 (99.4771)
Valid: 37 [ 300/390]  Loss: 0.5630 (0.412)  Acc@1: 82.8125 (86.3372)  Acc@5: 100.0000 (99.4342)
Valid: 37 [ 350/390]  Loss: 0.3918 (0.413)  Acc@1: 89.0625 (86.3470)  Acc@5: 100.0000 (99.4792)
Valid: 37 [ 390/390]  Loss: 0.3391 (0.412)  Acc@1: 87.5000 (86.3600)  Acc@5: 100.0000 (99.4920)
valid_acc 86.360000
epoch = 37   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3176, 0.6824],
        [0.3076, 0.6924],
        [0.3020, 0.6980],
        [0.2512, 0.7488],
        [0.3437, 0.6563],
        [0.4030, 0.5970],
        [0.3101, 0.6899],
        [0.4637, 0.5363],
        [0.4483, 0.5517],
        [0.3762, 0.6238],
        [0.4060, 0.5940],
        [0.4352, 0.5648],
        [0.4222, 0.5778],
        [0.4077, 0.5923]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.2217, 0.7783],
        [0.2333, 0.7667],
        [0.2628, 0.7372],
        [0.2705, 0.7295],
        [0.3452, 0.6548],
        [0.1860, 0.8140],
        [0.2429, 0.7571],
        [0.3462, 0.6538],
        [0.3371, 0.6629],
        [0.2375, 0.7625],
        [0.2882, 0.7118],
        [0.3451, 0.6549],
        [0.3639, 0.6361],
        [0.4005, 0.5995]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 38 [   0/390]  Loss: 0.1223 (0.122)  Acc@1: 95.3125 (95.3125)  Acc@5: 100.0000 (100.0000)LR: 4.252e-03
Train: 38 [  50/390]  Loss: 0.1304 (0.160)  Acc@1: 95.3125 (94.1789)  Acc@5: 100.0000 (100.0000)LR: 4.252e-03
Train: 38 [ 100/390]  Loss: 0.07628 (0.169)  Acc@1: 98.4375 (93.9666)  Acc@5: 100.0000 (99.9691)LR: 4.252e-03
Train: 38 [ 150/390]  Loss: 0.1022 (0.169)  Acc@1: 95.3125 (94.0087)  Acc@5: 100.0000 (99.9276)LR: 4.252e-03
Train: 38 [ 200/390]  Loss: 0.2113 (0.171)  Acc@1: 92.1875 (94.0065)  Acc@5: 100.0000 (99.9300)LR: 4.252e-03
Train: 38 [ 250/390]  Loss: 0.1072 (0.169)  Acc@1: 98.4375 (94.1733)  Acc@5: 100.0000 (99.9440)LR: 4.252e-03
Train: 38 [ 300/390]  Loss: 0.3201 (0.172)  Acc@1: 92.1875 (94.0303)  Acc@5: 100.0000 (99.9429)LR: 4.252e-03
Train: 38 [ 350/390]  Loss: 0.2254 (0.175)  Acc@1: 90.6250 (93.8925)  Acc@5: 100.0000 (99.9466)LR: 4.252e-03
Train: 38 [ 390/390]  Loss: 0.1812 (0.174)  Acc@1: 95.0000 (93.9320)  Acc@5: 100.0000 (99.9480)LR: 4.252e-03
train_acc 93.932000
Valid: 38 [   0/390]  Loss: 0.4270 (0.427)  Acc@1: 84.3750 (84.3750)  Acc@5: 100.0000 (100.0000)
Valid: 38 [  50/390]  Loss: 0.3242 (0.383)  Acc@1: 87.5000 (87.0404)  Acc@5: 100.0000 (99.5098)
Valid: 38 [ 100/390]  Loss: 0.3076 (0.393)  Acc@1: 89.0625 (86.5254)  Acc@5: 100.0000 (99.4740)
Valid: 38 [ 150/390]  Loss: 0.4001 (0.400)  Acc@1: 85.9375 (86.3307)  Acc@5: 100.0000 (99.4412)
Valid: 38 [ 200/390]  Loss: 0.2607 (0.404)  Acc@1: 90.6250 (86.2174)  Acc@5: 100.0000 (99.4170)
Valid: 38 [ 250/390]  Loss: 0.3750 (0.404)  Acc@1: 90.6250 (86.3421)  Acc@5: 96.8750 (99.4335)
Valid: 38 [ 300/390]  Loss: 0.3704 (0.402)  Acc@1: 89.0625 (86.5137)  Acc@5: 100.0000 (99.4446)
Valid: 38 [ 350/390]  Loss: 0.2483 (0.404)  Acc@1: 89.0625 (86.4895)  Acc@5: 100.0000 (99.4302)
Valid: 38 [ 390/390]  Loss: 0.4714 (0.406)  Acc@1: 80.0000 (86.4360)  Acc@5: 100.0000 (99.4400)
valid_acc 86.436000
epoch = 38   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3203, 0.6797],
        [0.3135, 0.6865],
        [0.2998, 0.7002],
        [0.2552, 0.7448],
        [0.3499, 0.6501],
        [0.4047, 0.5953],
        [0.3166, 0.6834],
        [0.4726, 0.5274],
        [0.4483, 0.5517],
        [0.3810, 0.6190],
        [0.4088, 0.5912],
        [0.4392, 0.5608],
        [0.4304, 0.5696],
        [0.4164, 0.5836]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.2222, 0.7778],
        [0.2296, 0.7704],
        [0.2644, 0.7356],
        [0.2695, 0.7305],
        [0.3426, 0.6574],
        [0.1817, 0.8183],
        [0.2416, 0.7584],
        [0.3454, 0.6546],
        [0.3335, 0.6665],
        [0.2307, 0.7693],
        [0.2857, 0.7143],
        [0.3439, 0.6561],
        [0.3611, 0.6389],
        [0.4062, 0.5938]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 39 [   0/390]  Loss: 0.2212 (0.221)  Acc@1: 93.7500 (93.7500)  Acc@5: 100.0000 (100.0000)LR: 3.754e-03
Train: 39 [  50/390]  Loss: 0.3156 (0.163)  Acc@1: 90.6250 (94.0257)  Acc@5: 100.0000 (99.9694)LR: 3.754e-03
Train: 39 [ 100/390]  Loss: 0.1276 (0.159)  Acc@1: 96.8750 (94.5699)  Acc@5: 100.0000 (99.9536)LR: 3.754e-03
Train: 39 [ 150/390]  Loss: 0.08142 (0.161)  Acc@1: 95.3125 (94.2260)  Acc@5: 100.0000 (99.9586)LR: 3.754e-03
Train: 39 [ 200/390]  Loss: 0.1261 (0.162)  Acc@1: 95.3125 (94.2164)  Acc@5: 100.0000 (99.9611)LR: 3.754e-03
Train: 39 [ 250/390]  Loss: 0.3502 (0.161)  Acc@1: 85.9375 (94.2916)  Acc@5: 100.0000 (99.9626)LR: 3.754e-03
Train: 39 [ 300/390]  Loss: 0.1387 (0.161)  Acc@1: 93.7500 (94.2431)  Acc@5: 100.0000 (99.9585)LR: 3.754e-03
Train: 39 [ 350/390]  Loss: 0.08382 (0.161)  Acc@1: 95.3125 (94.2753)  Acc@5: 100.0000 (99.9555)LR: 3.754e-03
Train: 39 [ 390/390]  Loss: 0.3168 (0.163)  Acc@1: 87.5000 (94.2520)  Acc@5: 100.0000 (99.9520)LR: 3.754e-03
train_acc 94.252000
Valid: 39 [   0/390]  Loss: 0.4951 (0.495)  Acc@1: 81.2500 (81.2500)  Acc@5: 100.0000 (100.0000)
Valid: 39 [  50/390]  Loss: 0.3267 (0.437)  Acc@1: 92.1875 (86.0600)  Acc@5: 100.0000 (99.3566)
Valid: 39 [ 100/390]  Loss: 0.3896 (0.429)  Acc@1: 85.9375 (86.1077)  Acc@5: 100.0000 (99.3502)
Valid: 39 [ 150/390]  Loss: 0.4675 (0.410)  Acc@1: 79.6875 (86.4756)  Acc@5: 100.0000 (99.4516)
Valid: 39 [ 200/390]  Loss: 0.6279 (0.417)  Acc@1: 78.1250 (86.3184)  Acc@5: 100.0000 (99.5025)
Valid: 39 [ 250/390]  Loss: 0.3877 (0.411)  Acc@1: 85.9375 (86.3546)  Acc@5: 98.4375 (99.4895)
Valid: 39 [ 300/390]  Loss: 0.5376 (0.410)  Acc@1: 82.8125 (86.3113)  Acc@5: 100.0000 (99.4861)
Valid: 39 [ 350/390]  Loss: 0.3599 (0.408)  Acc@1: 89.0625 (86.3827)  Acc@5: 98.4375 (99.5103)
Valid: 39 [ 390/390]  Loss: 0.3787 (0.410)  Acc@1: 87.5000 (86.3440)  Acc@5: 100.0000 (99.4640)
valid_acc 86.344000
epoch = 39   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3208, 0.6792],
        [0.3198, 0.6802],
        [0.3028, 0.6972],
        [0.2578, 0.7422],
        [0.3546, 0.6454],
        [0.4110, 0.5890],
        [0.3209, 0.6791],
        [0.4769, 0.5231],
        [0.4517, 0.5483],
        [0.3832, 0.6168],
        [0.4105, 0.5895],
        [0.4434, 0.5566],
        [0.4348, 0.5652],
        [0.4223, 0.5777]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.2234, 0.7766],
        [0.2268, 0.7732],
        [0.2646, 0.7354],
        [0.2663, 0.7337],
        [0.3416, 0.6584],
        [0.1815, 0.8185],
        [0.2364, 0.7636],
        [0.3462, 0.6538],
        [0.3335, 0.6665],
        [0.2281, 0.7719],
        [0.2859, 0.7141],
        [0.3509, 0.6491],
        [0.3599, 0.6401],
        [0.4018, 0.5982]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 40 [   0/390]  Loss: 0.09181 (0.0918)  Acc@1: 96.8750 (96.8750)  Acc@5: 100.0000 (100.0000)LR: 3.292e-03
Train: 40 [  50/390]  Loss: 0.1762 (0.159)  Acc@1: 95.3125 (94.5159)  Acc@5: 100.0000 (99.9387)LR: 3.292e-03
Train: 40 [ 100/390]  Loss: 0.1435 (0.155)  Acc@1: 93.7500 (94.6009)  Acc@5: 100.0000 (99.9072)LR: 3.292e-03
Train: 40 [ 150/390]  Loss: 0.1274 (0.156)  Acc@1: 95.3125 (94.5778)  Acc@5: 100.0000 (99.9379)LR: 3.292e-03
Train: 40 [ 200/390]  Loss: 0.1025 (0.155)  Acc@1: 95.3125 (94.6440)  Acc@5: 100.0000 (99.9223)LR: 3.292e-03
Train: 40 [ 250/390]  Loss: 0.05800 (0.156)  Acc@1: 100.0000 (94.5219)  Acc@5: 100.0000 (99.9315)LR: 3.292e-03
Train: 40 [ 300/390]  Loss: 0.1525 (0.156)  Acc@1: 92.1875 (94.5390)  Acc@5: 100.0000 (99.9377)LR: 3.292e-03
Train: 40 [ 350/390]  Loss: 0.1597 (0.157)  Acc@1: 95.3125 (94.5023)  Acc@5: 100.0000 (99.9466)LR: 3.292e-03
Train: 40 [ 390/390]  Loss: 0.06489 (0.157)  Acc@1: 97.5000 (94.4800)  Acc@5: 100.0000 (99.9480)LR: 3.292e-03
train_acc 94.480000
Valid: 40 [   0/390]  Loss: 0.4949 (0.495)  Acc@1: 81.2500 (81.2500)  Acc@5: 98.4375 (98.4375)
Valid: 40 [  50/390]  Loss: 0.4944 (0.436)  Acc@1: 90.6250 (85.2635)  Acc@5: 98.4375 (99.4179)
Valid: 40 [ 100/390]  Loss: 0.8345 (0.438)  Acc@1: 76.5625 (85.5507)  Acc@5: 95.3125 (99.2574)
Valid: 40 [ 150/390]  Loss: 0.3918 (0.430)  Acc@1: 85.9375 (85.5339)  Acc@5: 100.0000 (99.3274)
Valid: 40 [ 200/390]  Loss: 0.4431 (0.440)  Acc@1: 87.5000 (85.3389)  Acc@5: 98.4375 (99.3315)
Valid: 40 [ 250/390]  Loss: 0.6855 (0.439)  Acc@1: 76.5625 (85.3399)  Acc@5: 100.0000 (99.3526)
Valid: 40 [ 300/390]  Loss: 0.3611 (0.437)  Acc@1: 84.3750 (85.4651)  Acc@5: 100.0000 (99.3667)
Valid: 40 [ 350/390]  Loss: 0.6592 (0.437)  Acc@1: 81.2500 (85.4745)  Acc@5: 95.3125 (99.3679)
Valid: 40 [ 390/390]  Loss: 0.2615 (0.434)  Acc@1: 92.5000 (85.5280)  Acc@5: 100.0000 (99.3920)
valid_acc 85.528000
epoch = 40   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3222, 0.6778],
        [0.3206, 0.6794],
        [0.3083, 0.6917],
        [0.2635, 0.7365],
        [0.3598, 0.6402],
        [0.4146, 0.5854],
        [0.3252, 0.6748],
        [0.4859, 0.5141],
        [0.4569, 0.5431],
        [0.3824, 0.6176],
        [0.4179, 0.5821],
        [0.4510, 0.5490],
        [0.4455, 0.5545],
        [0.4310, 0.5690]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.2251, 0.7749],
        [0.2238, 0.7762],
        [0.2679, 0.7321],
        [0.2604, 0.7396],
        [0.3389, 0.6611],
        [0.1820, 0.8180],
        [0.2324, 0.7676],
        [0.3428, 0.6572],
        [0.3328, 0.6672],
        [0.2242, 0.7758],
        [0.2814, 0.7186],
        [0.3497, 0.6503],
        [0.3578, 0.6422],
        [0.4065, 0.5935]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 41 [   0/390]  Loss: 0.1264 (0.126)  Acc@1: 93.7500 (93.7500)  Acc@5: 100.0000 (100.0000)LR: 2.868e-03
Train: 41 [  50/390]  Loss: 0.2088 (0.140)  Acc@1: 95.3125 (95.2206)  Acc@5: 100.0000 (99.9081)LR: 2.868e-03
Train: 41 [ 100/390]  Loss: 0.1014 (0.143)  Acc@1: 95.3125 (94.8175)  Acc@5: 100.0000 (99.9226)LR: 2.868e-03
Train: 41 [ 150/390]  Loss: 0.1103 (0.145)  Acc@1: 95.3125 (94.8262)  Acc@5: 100.0000 (99.9483)LR: 2.868e-03
Train: 41 [ 200/390]  Loss: 0.1782 (0.146)  Acc@1: 92.1875 (94.7528)  Acc@5: 100.0000 (99.9611)LR: 2.868e-03
Train: 41 [ 250/390]  Loss: 0.2156 (0.143)  Acc@1: 95.3125 (94.8705)  Acc@5: 98.4375 (99.9626)LR: 2.868e-03
Train: 41 [ 300/390]  Loss: 0.1173 (0.143)  Acc@1: 95.3125 (94.8401)  Acc@5: 100.0000 (99.9637)LR: 2.868e-03
Train: 41 [ 350/390]  Loss: 0.1114 (0.146)  Acc@1: 95.3125 (94.7561)  Acc@5: 100.0000 (99.9599)LR: 2.868e-03
Train: 41 [ 390/390]  Loss: 0.1317 (0.145)  Acc@1: 95.0000 (94.8280)  Acc@5: 100.0000 (99.9640)LR: 2.868e-03
train_acc 94.828000
Valid: 41 [   0/390]  Loss: 0.4377 (0.438)  Acc@1: 89.0625 (89.0625)  Acc@5: 100.0000 (100.0000)
Valid: 41 [  50/390]  Loss: 0.4270 (0.416)  Acc@1: 81.2500 (86.5502)  Acc@5: 100.0000 (99.5711)
Valid: 41 [ 100/390]  Loss: 0.2795 (0.406)  Acc@1: 92.1875 (86.6955)  Acc@5: 100.0000 (99.4431)
Valid: 41 [ 150/390]  Loss: 0.5137 (0.404)  Acc@1: 87.5000 (86.8377)  Acc@5: 98.4375 (99.4412)
Valid: 41 [ 200/390]  Loss: 0.4089 (0.404)  Acc@1: 82.8125 (86.6682)  Acc@5: 100.0000 (99.4792)
Valid: 41 [ 250/390]  Loss: 0.4158 (0.402)  Acc@1: 84.3750 (86.7841)  Acc@5: 98.4375 (99.5020)
Valid: 41 [ 300/390]  Loss: 0.4106 (0.410)  Acc@1: 84.3750 (86.5708)  Acc@5: 100.0000 (99.4757)
Valid: 41 [ 350/390]  Loss: 0.2898 (0.412)  Acc@1: 87.5000 (86.5518)  Acc@5: 100.0000 (99.4658)
Valid: 41 [ 390/390]  Loss: 0.3650 (0.411)  Acc@1: 85.0000 (86.5480)  Acc@5: 100.0000 (99.4760)
valid_acc 86.548000
epoch = 41   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3225, 0.6775],
        [0.3255, 0.6745],
        [0.3116, 0.6884],
        [0.2705, 0.7295],
        [0.3666, 0.6334],
        [0.4190, 0.5810],
        [0.3315, 0.6685],
        [0.4912, 0.5088],
        [0.4634, 0.5366],
        [0.3827, 0.6173],
        [0.4288, 0.5712],
        [0.4576, 0.5424],
        [0.4531, 0.5469],
        [0.4363, 0.5637]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.2278, 0.7722],
        [0.2223, 0.7777],
        [0.2643, 0.7357],
        [0.2552, 0.7448],
        [0.3361, 0.6639],
        [0.1797, 0.8203],
        [0.2317, 0.7683],
        [0.3402, 0.6598],
        [0.3340, 0.6659],
        [0.2208, 0.7792],
        [0.2824, 0.7176],
        [0.3466, 0.6534],
        [0.3568, 0.6432],
        [0.4133, 0.5867]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 42 [   0/390]  Loss: 0.1055 (0.105)  Acc@1: 96.8750 (96.8750)  Acc@5: 100.0000 (100.0000)LR: 2.484e-03
Train: 42 [  50/390]  Loss: 0.1377 (0.135)  Acc@1: 93.7500 (95.2512)  Acc@5: 100.0000 (99.9694)LR: 2.484e-03
Train: 42 [ 100/390]  Loss: 0.1853 (0.141)  Acc@1: 92.1875 (95.0804)  Acc@5: 100.0000 (99.9381)LR: 2.484e-03
Train: 42 [ 150/390]  Loss: 0.09797 (0.137)  Acc@1: 98.4375 (95.2608)  Acc@5: 100.0000 (99.9586)LR: 2.484e-03
Train: 42 [ 200/390]  Loss: 0.1518 (0.144)  Acc@1: 93.7500 (95.0093)  Acc@5: 100.0000 (99.9689)LR: 2.484e-03
Train: 42 [ 250/390]  Loss: 0.06953 (0.142)  Acc@1: 98.4375 (95.0822)  Acc@5: 100.0000 (99.9564)LR: 2.484e-03
Train: 42 [ 300/390]  Loss: 0.1397 (0.143)  Acc@1: 96.8750 (95.0114)  Acc@5: 100.0000 (99.9637)LR: 2.484e-03
Train: 42 [ 350/390]  Loss: 0.1309 (0.143)  Acc@1: 95.3125 (94.9519)  Acc@5: 100.0000 (99.9599)LR: 2.484e-03
Train: 42 [ 390/390]  Loss: 0.2809 (0.145)  Acc@1: 85.0000 (94.8960)  Acc@5: 100.0000 (99.9600)LR: 2.484e-03
train_acc 94.896000
Valid: 42 [   0/390]  Loss: 0.2310 (0.231)  Acc@1: 95.3125 (95.3125)  Acc@5: 100.0000 (100.0000)
Valid: 42 [  50/390]  Loss: 0.3459 (0.421)  Acc@1: 89.0625 (86.9485)  Acc@5: 100.0000 (99.3566)
Valid: 42 [ 100/390]  Loss: 0.2568 (0.409)  Acc@1: 92.1875 (87.0359)  Acc@5: 100.0000 (99.3967)
Valid: 42 [ 150/390]  Loss: 0.3501 (0.406)  Acc@1: 89.0625 (87.0137)  Acc@5: 100.0000 (99.4205)
Valid: 42 [ 200/390]  Loss: 0.4944 (0.410)  Acc@1: 81.2500 (86.8004)  Acc@5: 100.0000 (99.4170)
Valid: 42 [ 250/390]  Loss: 0.1858 (0.408)  Acc@1: 96.8750 (86.8588)  Acc@5: 100.0000 (99.4335)
Valid: 42 [ 300/390]  Loss: 0.1925 (0.409)  Acc@1: 90.6250 (86.8667)  Acc@5: 100.0000 (99.4082)
Valid: 42 [ 350/390]  Loss: 0.5811 (0.410)  Acc@1: 81.2500 (86.7788)  Acc@5: 98.4375 (99.4257)
Valid: 42 [ 390/390]  Loss: 0.5337 (0.408)  Acc@1: 80.0000 (86.7760)  Acc@5: 97.5000 (99.4440)
valid_acc 86.776000
epoch = 42   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3292, 0.6708],
        [0.3296, 0.6704],
        [0.3098, 0.6902],
        [0.2724, 0.7276],
        [0.3743, 0.6257],
        [0.4266, 0.5734],
        [0.3356, 0.6644],
        [0.5022, 0.4978],
        [0.4688, 0.5312],
        [0.3852, 0.6148],
        [0.4315, 0.5685],
        [0.4682, 0.5318],
        [0.4608, 0.5392],
        [0.4451, 0.5549]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.2264, 0.7736],
        [0.2228, 0.7772],
        [0.2606, 0.7394],
        [0.2528, 0.7472],
        [0.3357, 0.6643],
        [0.1788, 0.8212],
        [0.2261, 0.7739],
        [0.3365, 0.6635],
        [0.3326, 0.6674],
        [0.2203, 0.7797],
        [0.2836, 0.7164],
        [0.3440, 0.6560],
        [0.3537, 0.6463],
        [0.4166, 0.5834]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 43 [   0/390]  Loss: 0.1135 (0.113)  Acc@1: 96.8750 (96.8750)  Acc@5: 100.0000 (100.0000)LR: 2.142e-03
Train: 43 [  50/390]  Loss: 0.07911 (0.128)  Acc@1: 96.8750 (95.4657)  Acc@5: 100.0000 (99.9387)LR: 2.142e-03
Train: 43 [ 100/390]  Loss: 0.2371 (0.127)  Acc@1: 93.7500 (95.7766)  Acc@5: 100.0000 (99.9691)LR: 2.142e-03
Train: 43 [ 150/390]  Loss: 0.1397 (0.129)  Acc@1: 96.8750 (95.6540)  Acc@5: 100.0000 (99.9586)LR: 2.142e-03
Train: 43 [ 200/390]  Loss: 0.1060 (0.133)  Acc@1: 95.3125 (95.4213)  Acc@5: 100.0000 (99.9456)LR: 2.142e-03
Train: 43 [ 250/390]  Loss: 0.1341 (0.133)  Acc@1: 96.8750 (95.3872)  Acc@5: 100.0000 (99.9502)LR: 2.142e-03
Train: 43 [ 300/390]  Loss: 0.1989 (0.135)  Acc@1: 90.6250 (95.2969)  Acc@5: 100.0000 (99.9533)LR: 2.142e-03
Train: 43 [ 350/390]  Loss: 0.09591 (0.135)  Acc@1: 96.8750 (95.2724)  Acc@5: 100.0000 (99.9599)LR: 2.142e-03
Train: 43 [ 390/390]  Loss: 0.2070 (0.136)  Acc@1: 95.0000 (95.2400)  Acc@5: 100.0000 (99.9640)LR: 2.142e-03
train_acc 95.240000
Valid: 43 [   0/390]  Loss: 0.5879 (0.588)  Acc@1: 81.2500 (81.2500)  Acc@5: 100.0000 (100.0000)
Valid: 43 [  50/390]  Loss: 0.6196 (0.412)  Acc@1: 82.8125 (86.6728)  Acc@5: 98.4375 (99.4179)
Valid: 43 [ 100/390]  Loss: 0.3115 (0.404)  Acc@1: 85.9375 (86.8038)  Acc@5: 100.0000 (99.4895)
Valid: 43 [ 150/390]  Loss: 0.4504 (0.411)  Acc@1: 87.5000 (86.5377)  Acc@5: 98.4375 (99.4930)
Valid: 43 [ 200/390]  Loss: 0.4021 (0.414)  Acc@1: 82.8125 (86.5516)  Acc@5: 100.0000 (99.4869)
Valid: 43 [ 250/390]  Loss: 0.3779 (0.415)  Acc@1: 90.6250 (86.5102)  Acc@5: 100.0000 (99.4895)
Valid: 43 [ 300/390]  Loss: 0.2184 (0.415)  Acc@1: 95.3125 (86.4981)  Acc@5: 100.0000 (99.4757)
Valid: 43 [ 350/390]  Loss: 0.2981 (0.418)  Acc@1: 82.8125 (86.3827)  Acc@5: 98.4375 (99.4925)
Valid: 43 [ 390/390]  Loss: 0.5176 (0.418)  Acc@1: 85.0000 (86.3840)  Acc@5: 100.0000 (99.4880)
valid_acc 86.384000
epoch = 43   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3320, 0.6680],
        [0.3345, 0.6655],
        [0.3109, 0.6891],
        [0.2756, 0.7244],
        [0.3789, 0.6211],
        [0.4307, 0.5693],
        [0.3440, 0.6560],
        [0.5057, 0.4943],
        [0.4793, 0.5207],
        [0.3858, 0.6142],
        [0.4344, 0.5656],
        [0.4682, 0.5318],
        [0.4662, 0.5338],
        [0.4512, 0.5488]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.2288, 0.7712],
        [0.2205, 0.7795],
        [0.2574, 0.7426],
        [0.2528, 0.7472],
        [0.3313, 0.6687],
        [0.1767, 0.8233],
        [0.2196, 0.7804],
        [0.3327, 0.6673],
        [0.3317, 0.6683],
        [0.2191, 0.7809],
        [0.2881, 0.7119],
        [0.3453, 0.6547],
        [0.3555, 0.6445],
        [0.4148, 0.5852]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 44 [   0/390]  Loss: 0.1470 (0.147)  Acc@1: 93.7500 (93.7500)  Acc@5: 100.0000 (100.0000)LR: 1.843e-03
Train: 44 [  50/390]  Loss: 0.09189 (0.135)  Acc@1: 98.4375 (94.7917)  Acc@5: 100.0000 (99.9387)LR: 1.843e-03
Train: 44 [ 100/390]  Loss: 0.1598 (0.126)  Acc@1: 93.7500 (95.3125)  Acc@5: 100.0000 (99.9536)LR: 1.843e-03
Train: 44 [ 150/390]  Loss: 0.08614 (0.127)  Acc@1: 96.8750 (95.3539)  Acc@5: 100.0000 (99.9379)LR: 1.843e-03
Train: 44 [ 200/390]  Loss: 0.05226 (0.126)  Acc@1: 98.4375 (95.4524)  Acc@5: 100.0000 (99.9378)LR: 1.843e-03
Train: 44 [ 250/390]  Loss: 0.1850 (0.126)  Acc@1: 93.7500 (95.4930)  Acc@5: 100.0000 (99.9253)LR: 1.843e-03
Train: 44 [ 300/390]  Loss: 0.03556 (0.126)  Acc@1: 98.4375 (95.5098)  Acc@5: 100.0000 (99.9377)LR: 1.843e-03
Train: 44 [ 350/390]  Loss: 0.09933 (0.128)  Acc@1: 96.8750 (95.4282)  Acc@5: 100.0000 (99.9421)LR: 1.843e-03
Train: 44 [ 390/390]  Loss: 0.1741 (0.128)  Acc@1: 92.5000 (95.4000)  Acc@5: 100.0000 (99.9440)LR: 1.843e-03
train_acc 95.400000
Valid: 44 [   0/390]  Loss: 0.5405 (0.541)  Acc@1: 85.9375 (85.9375)  Acc@5: 98.4375 (98.4375)
Valid: 44 [  50/390]  Loss: 0.3816 (0.431)  Acc@1: 82.8125 (85.7843)  Acc@5: 98.4375 (99.5098)
Valid: 44 [ 100/390]  Loss: 0.3118 (0.422)  Acc@1: 87.5000 (86.2933)  Acc@5: 100.0000 (99.3502)
Valid: 44 [ 150/390]  Loss: 0.3528 (0.430)  Acc@1: 87.5000 (86.0203)  Acc@5: 100.0000 (99.4205)
Valid: 44 [ 200/390]  Loss: 0.4297 (0.422)  Acc@1: 82.8125 (86.3650)  Acc@5: 100.0000 (99.4481)
Valid: 44 [ 250/390]  Loss: 0.3860 (0.419)  Acc@1: 82.8125 (86.4231)  Acc@5: 100.0000 (99.4460)
Valid: 44 [ 300/390]  Loss: 0.2720 (0.418)  Acc@1: 92.1875 (86.3839)  Acc@5: 98.4375 (99.4186)
Valid: 44 [ 350/390]  Loss: 0.5435 (0.417)  Acc@1: 90.6250 (86.4316)  Acc@5: 95.3125 (99.3901)
Valid: 44 [ 390/390]  Loss: 0.6748 (0.418)  Acc@1: 75.0000 (86.4840)  Acc@5: 100.0000 (99.4040)
valid_acc 86.484000
epoch = 44   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3331, 0.6669],
        [0.3354, 0.6646],
        [0.3142, 0.6858],
        [0.2822, 0.7178],
        [0.3810, 0.6190],
        [0.4323, 0.5677],
        [0.3491, 0.6509],
        [0.5169, 0.4831],
        [0.4877, 0.5123],
        [0.3878, 0.6122],
        [0.4413, 0.5587],
        [0.4710, 0.5290],
        [0.4711, 0.5289],
        [0.4578, 0.5422]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.2290, 0.7710],
        [0.2167, 0.7833],
        [0.2577, 0.7423],
        [0.2440, 0.7560],
        [0.3278, 0.6722],
        [0.1745, 0.8255],
        [0.2150, 0.7850],
        [0.3321, 0.6679],
        [0.3303, 0.6697],
        [0.2170, 0.7830],
        [0.2881, 0.7119],
        [0.3419, 0.6581],
        [0.3547, 0.6453],
        [0.4165, 0.5835]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 45 [   0/390]  Loss: 0.2151 (0.215)  Acc@1: 93.7500 (93.7500)  Acc@5: 100.0000 (100.0000)LR: 1.587e-03
Train: 45 [  50/390]  Loss: 0.08764 (0.128)  Acc@1: 95.3125 (95.3431)  Acc@5: 100.0000 (99.9387)LR: 1.587e-03
Train: 45 [ 100/390]  Loss: 0.1690 (0.125)  Acc@1: 95.3125 (95.5291)  Acc@5: 100.0000 (99.9536)LR: 1.587e-03
Train: 45 [ 150/390]  Loss: 0.1495 (0.123)  Acc@1: 95.3125 (95.7057)  Acc@5: 100.0000 (99.9379)LR: 1.587e-03
Train: 45 [ 200/390]  Loss: 0.09671 (0.121)  Acc@1: 95.3125 (95.7711)  Acc@5: 100.0000 (99.9534)LR: 1.587e-03
Train: 45 [ 250/390]  Loss: 0.1072 (0.123)  Acc@1: 96.8750 (95.7420)  Acc@5: 100.0000 (99.9502)LR: 1.587e-03
Train: 45 [ 300/390]  Loss: 0.1473 (0.123)  Acc@1: 98.4375 (95.7122)  Acc@5: 100.0000 (99.9585)LR: 1.587e-03
Train: 45 [ 350/390]  Loss: 0.08551 (0.123)  Acc@1: 96.8750 (95.7354)  Acc@5: 100.0000 (99.9599)LR: 1.587e-03
Train: 45 [ 390/390]  Loss: 0.1240 (0.123)  Acc@1: 95.0000 (95.6720)  Acc@5: 100.0000 (99.9640)LR: 1.587e-03
train_acc 95.672000
Valid: 45 [   0/390]  Loss: 0.5205 (0.521)  Acc@1: 87.5000 (87.5000)  Acc@5: 98.4375 (98.4375)
Valid: 45 [  50/390]  Loss: 0.4458 (0.432)  Acc@1: 81.2500 (86.2439)  Acc@5: 100.0000 (99.1115)
Valid: 45 [ 100/390]  Loss: 0.5195 (0.430)  Acc@1: 81.2500 (86.1541)  Acc@5: 100.0000 (99.2729)
Valid: 45 [ 150/390]  Loss: 0.3518 (0.435)  Acc@1: 81.2500 (85.9582)  Acc@5: 100.0000 (99.3171)
Valid: 45 [ 200/390]  Loss: 0.5977 (0.444)  Acc@1: 82.8125 (85.6499)  Acc@5: 98.4375 (99.2615)
Valid: 45 [ 250/390]  Loss: 0.6362 (0.439)  Acc@1: 81.2500 (85.7819)  Acc@5: 100.0000 (99.3215)
Valid: 45 [ 300/390]  Loss: 0.4929 (0.431)  Acc@1: 85.9375 (86.0569)  Acc@5: 98.4375 (99.3615)
Valid: 45 [ 350/390]  Loss: 0.3833 (0.432)  Acc@1: 82.8125 (86.0310)  Acc@5: 98.4375 (99.3545)
Valid: 45 [ 390/390]  Loss: 0.4597 (0.433)  Acc@1: 87.5000 (85.9840)  Acc@5: 100.0000 (99.3640)
valid_acc 85.984000
epoch = 45   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3363, 0.6637],
        [0.3377, 0.6623],
        [0.3170, 0.6830],
        [0.2880, 0.7120],
        [0.3813, 0.6187],
        [0.4337, 0.5663],
        [0.3562, 0.6438],
        [0.5240, 0.4760],
        [0.4938, 0.5062],
        [0.3874, 0.6126],
        [0.4462, 0.5538],
        [0.4737, 0.5263],
        [0.4866, 0.5134],
        [0.4656, 0.5344]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.2278, 0.7722],
        [0.2148, 0.7852],
        [0.2557, 0.7443],
        [0.2411, 0.7589],
        [0.3230, 0.6770],
        [0.1750, 0.8250],
        [0.2120, 0.7880],
        [0.3275, 0.6725],
        [0.3287, 0.6713],
        [0.2149, 0.7851],
        [0.2886, 0.7114],
        [0.3428, 0.6572],
        [0.3563, 0.6437],
        [0.4173, 0.5827]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 46 [   0/390]  Loss: 0.1735 (0.173)  Acc@1: 90.6250 (90.6250)  Acc@5: 100.0000 (100.0000)LR: 1.377e-03
Train: 46 [  50/390]  Loss: 0.1570 (0.119)  Acc@1: 95.3125 (95.9559)  Acc@5: 100.0000 (99.9387)LR: 1.377e-03
Train: 46 [ 100/390]  Loss: 0.09741 (0.126)  Acc@1: 95.3125 (95.6528)  Acc@5: 100.0000 (99.9381)LR: 1.377e-03
Train: 46 [ 150/390]  Loss: 0.06115 (0.127)  Acc@1: 96.8750 (95.4574)  Acc@5: 100.0000 (99.9483)LR: 1.377e-03
Train: 46 [ 200/390]  Loss: 0.04658 (0.126)  Acc@1: 98.4375 (95.4835)  Acc@5: 100.0000 (99.9611)LR: 1.377e-03
Train: 46 [ 250/390]  Loss: 0.03259 (0.123)  Acc@1: 100.0000 (95.5677)  Acc@5: 100.0000 (99.9626)LR: 1.377e-03
Train: 46 [ 300/390]  Loss: 0.1357 (0.124)  Acc@1: 93.7500 (95.5617)  Acc@5: 100.0000 (99.9689)LR: 1.377e-03
Train: 46 [ 350/390]  Loss: 0.2423 (0.125)  Acc@1: 87.5000 (95.4995)  Acc@5: 100.0000 (99.9688)LR: 1.377e-03
Train: 46 [ 390/390]  Loss: 0.1534 (0.124)  Acc@1: 95.0000 (95.5560)  Acc@5: 100.0000 (99.9720)LR: 1.377e-03
train_acc 95.556000
Valid: 46 [   0/390]  Loss: 0.4495 (0.449)  Acc@1: 84.3750 (84.3750)  Acc@5: 100.0000 (100.0000)
Valid: 46 [  50/390]  Loss: 0.7456 (0.445)  Acc@1: 78.1250 (85.6924)  Acc@5: 100.0000 (99.2953)
Valid: 46 [ 100/390]  Loss: 0.3281 (0.448)  Acc@1: 87.5000 (85.3342)  Acc@5: 100.0000 (99.3657)
Valid: 46 [ 150/390]  Loss: 0.4148 (0.434)  Acc@1: 87.5000 (85.6581)  Acc@5: 100.0000 (99.3895)
Valid: 46 [ 200/390]  Loss: 0.4553 (0.436)  Acc@1: 84.3750 (85.7587)  Acc@5: 98.4375 (99.4170)
Valid: 46 [ 250/390]  Loss: 0.4763 (0.431)  Acc@1: 85.9375 (85.9064)  Acc@5: 98.4375 (99.4335)
Valid: 46 [ 300/390]  Loss: 0.4045 (0.433)  Acc@1: 87.5000 (85.7558)  Acc@5: 100.0000 (99.4342)
Valid: 46 [ 350/390]  Loss: 0.5005 (0.427)  Acc@1: 79.6875 (85.9197)  Acc@5: 100.0000 (99.4347)
Valid: 46 [ 390/390]  Loss: 0.3569 (0.426)  Acc@1: 90.0000 (86.0040)  Acc@5: 100.0000 (99.4360)
valid_acc 86.004000
epoch = 46   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3367, 0.6633],
        [0.3423, 0.6577],
        [0.3202, 0.6798],
        [0.2906, 0.7094],
        [0.3834, 0.6166],
        [0.4350, 0.5650],
        [0.3599, 0.6401],
        [0.5306, 0.4694],
        [0.4985, 0.5015],
        [0.3915, 0.6085],
        [0.4507, 0.5493],
        [0.4813, 0.5187],
        [0.4961, 0.5039],
        [0.4727, 0.5273]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.2264, 0.7736],
        [0.2136, 0.7864],
        [0.2548, 0.7452],
        [0.2392, 0.7608],
        [0.3174, 0.6826],
        [0.1712, 0.8288],
        [0.2076, 0.7924],
        [0.3266, 0.6734],
        [0.3229, 0.6771],
        [0.2114, 0.7886],
        [0.2898, 0.7102],
        [0.3431, 0.6569],
        [0.3593, 0.6407],
        [0.4197, 0.5803]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 47 [   0/390]  Loss: 0.04170 (0.0417)  Acc@1: 100.0000 (100.0000)  Acc@5: 100.0000 (100.0000)LR: 1.213e-03
Train: 47 [  50/390]  Loss: 0.07906 (0.111)  Acc@1: 95.3125 (96.2929)  Acc@5: 100.0000 (99.8775)LR: 1.213e-03
Train: 47 [ 100/390]  Loss: 0.1436 (0.118)  Acc@1: 93.7500 (95.9777)  Acc@5: 100.0000 (99.9226)LR: 1.213e-03
Train: 47 [ 150/390]  Loss: 0.04630 (0.118)  Acc@1: 98.4375 (95.9230)  Acc@5: 100.0000 (99.9483)LR: 1.213e-03
Train: 47 [ 200/390]  Loss: 0.09878 (0.117)  Acc@1: 96.8750 (95.9810)  Acc@5: 100.0000 (99.9611)LR: 1.213e-03
Train: 47 [ 250/390]  Loss: 0.1426 (0.118)  Acc@1: 95.3125 (95.9475)  Acc@5: 100.0000 (99.9689)LR: 1.213e-03
Train: 47 [ 300/390]  Loss: 0.05007 (0.120)  Acc@1: 98.4375 (95.9095)  Acc@5: 100.0000 (99.9637)LR: 1.213e-03
Train: 47 [ 350/390]  Loss: 0.1187 (0.120)  Acc@1: 96.8750 (95.8868)  Acc@5: 100.0000 (99.9644)LR: 1.213e-03
Train: 47 [ 390/390]  Loss: 0.03567 (0.120)  Acc@1: 97.5000 (95.8640)  Acc@5: 100.0000 (99.9640)LR: 1.213e-03
train_acc 95.864000
Valid: 47 [   0/390]  Loss: 0.4937 (0.494)  Acc@1: 89.0625 (89.0625)  Acc@5: 100.0000 (100.0000)
Valid: 47 [  50/390]  Loss: 0.4133 (0.420)  Acc@1: 84.3750 (85.6311)  Acc@5: 100.0000 (99.4179)
Valid: 47 [ 100/390]  Loss: 0.4192 (0.422)  Acc@1: 89.0625 (86.2469)  Acc@5: 98.4375 (99.4740)
Valid: 47 [ 150/390]  Loss: 0.3777 (0.417)  Acc@1: 82.8125 (86.2376)  Acc@5: 98.4375 (99.4309)
Valid: 47 [ 200/390]  Loss: 0.2128 (0.413)  Acc@1: 89.0625 (86.2407)  Acc@5: 100.0000 (99.4558)
Valid: 47 [ 250/390]  Loss: 0.3335 (0.414)  Acc@1: 87.5000 (86.1927)  Acc@5: 100.0000 (99.4086)
Valid: 47 [ 300/390]  Loss: 0.2310 (0.417)  Acc@1: 95.3125 (86.2022)  Acc@5: 100.0000 (99.3978)
Valid: 47 [ 350/390]  Loss: 0.4453 (0.421)  Acc@1: 87.5000 (86.0978)  Acc@5: 100.0000 (99.4168)
Valid: 47 [ 390/390]  Loss: 0.5762 (0.424)  Acc@1: 82.5000 (86.1280)  Acc@5: 97.5000 (99.4160)
valid_acc 86.128000
epoch = 47   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3367, 0.6633],
        [0.3467, 0.6533],
        [0.3211, 0.6789],
        [0.2935, 0.7065],
        [0.3872, 0.6128],
        [0.4427, 0.5573],
        [0.3660, 0.6340],
        [0.5340, 0.4660],
        [0.5036, 0.4964],
        [0.3951, 0.6049],
        [0.4520, 0.5480],
        [0.4876, 0.5124],
        [0.5052, 0.4948],
        [0.4810, 0.5190]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.2286, 0.7714],
        [0.2094, 0.7906],
        [0.2523, 0.7477],
        [0.2398, 0.7602],
        [0.3178, 0.6822],
        [0.1695, 0.8305],
        [0.2048, 0.7952],
        [0.3265, 0.6735],
        [0.3231, 0.6769],
        [0.2090, 0.7910],
        [0.2883, 0.7117],
        [0.3411, 0.6589],
        [0.3595, 0.6405],
        [0.4181, 0.5819]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 48 [   0/390]  Loss: 0.05830 (0.0583)  Acc@1: 98.4375 (98.4375)  Acc@5: 100.0000 (100.0000)LR: 1.095e-03
Train: 48 [  50/390]  Loss: 0.1186 (0.124)  Acc@1: 95.3125 (95.5882)  Acc@5: 100.0000 (99.9694)LR: 1.095e-03
Train: 48 [ 100/390]  Loss: 0.1028 (0.118)  Acc@1: 95.3125 (95.6683)  Acc@5: 100.0000 (99.9845)LR: 1.095e-03
Train: 48 [ 150/390]  Loss: 0.07533 (0.120)  Acc@1: 96.8750 (95.6436)  Acc@5: 100.0000 (99.9793)LR: 1.095e-03
Train: 48 [ 200/390]  Loss: 0.1624 (0.122)  Acc@1: 95.3125 (95.5846)  Acc@5: 100.0000 (99.9767)LR: 1.095e-03
Train: 48 [ 250/390]  Loss: 0.2285 (0.120)  Acc@1: 95.3125 (95.7171)  Acc@5: 100.0000 (99.9813)LR: 1.095e-03
Train: 48 [ 300/390]  Loss: 0.2454 (0.122)  Acc@1: 92.1875 (95.6707)  Acc@5: 100.0000 (99.9844)LR: 1.095e-03
Train: 48 [ 350/390]  Loss: 0.1243 (0.122)  Acc@1: 96.8750 (95.6464)  Acc@5: 100.0000 (99.9866)LR: 1.095e-03
Train: 48 [ 390/390]  Loss: 0.1335 (0.122)  Acc@1: 95.0000 (95.6440)  Acc@5: 100.0000 (99.9840)LR: 1.095e-03
train_acc 95.644000
Valid: 48 [   0/390]  Loss: 0.5317 (0.532)  Acc@1: 84.3750 (84.3750)  Acc@5: 100.0000 (100.0000)
Valid: 48 [  50/390]  Loss: 0.3162 (0.447)  Acc@1: 87.5000 (85.0797)  Acc@5: 100.0000 (99.4485)
Valid: 48 [ 100/390]  Loss: 0.2827 (0.440)  Acc@1: 89.0625 (85.4889)  Acc@5: 98.4375 (99.4121)
Valid: 48 [ 150/390]  Loss: 0.4595 (0.432)  Acc@1: 81.2500 (85.5339)  Acc@5: 98.4375 (99.4619)
Valid: 48 [ 200/390]  Loss: 0.6577 (0.432)  Acc@1: 76.5625 (85.6266)  Acc@5: 98.4375 (99.4403)
Valid: 48 [ 250/390]  Loss: 0.1805 (0.429)  Acc@1: 93.7500 (85.7756)  Acc@5: 100.0000 (99.4709)
Valid: 48 [ 300/390]  Loss: 0.3933 (0.431)  Acc@1: 85.9375 (85.8025)  Acc@5: 100.0000 (99.4394)
Valid: 48 [ 350/390]  Loss: 0.2722 (0.432)  Acc@1: 90.6250 (85.8129)  Acc@5: 100.0000 (99.4124)
Valid: 48 [ 390/390]  Loss: 0.1967 (0.430)  Acc@1: 95.0000 (85.8520)  Acc@5: 100.0000 (99.4240)
valid_acc 85.852000
epoch = 48   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3398, 0.6602],
        [0.3509, 0.6491],
        [0.3218, 0.6782],
        [0.2966, 0.7034],
        [0.3880, 0.6120],
        [0.4459, 0.5541],
        [0.3720, 0.6280],
        [0.5435, 0.4565],
        [0.5085, 0.4915],
        [0.4024, 0.5976],
        [0.4532, 0.5468],
        [0.4951, 0.5049],
        [0.5080, 0.4920],
        [0.4847, 0.5153]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.2300, 0.7700],
        [0.2074, 0.7926],
        [0.2494, 0.7506],
        [0.2387, 0.7613],
        [0.3192, 0.6808],
        [0.1671, 0.8329],
        [0.2065, 0.7935],
        [0.3251, 0.6749],
        [0.3195, 0.6805],
        [0.2057, 0.7943],
        [0.2826, 0.7174],
        [0.3400, 0.6600],
        [0.3598, 0.6402],
        [0.4217, 0.5783]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 49 [   0/390]  Loss: 0.07437 (0.0744)  Acc@1: 98.4375 (98.4375)  Acc@5: 100.0000 (100.0000)LR: 1.024e-03
Train: 49 [  50/390]  Loss: 0.08853 (0.114)  Acc@1: 96.8750 (96.0784)  Acc@5: 100.0000 (100.0000)LR: 1.024e-03
Train: 49 [ 100/390]  Loss: 0.08356 (0.114)  Acc@1: 96.8750 (96.0241)  Acc@5: 100.0000 (100.0000)LR: 1.024e-03
Train: 49 [ 150/390]  Loss: 0.08329 (0.112)  Acc@1: 96.8750 (96.1300)  Acc@5: 100.0000 (99.9793)LR: 1.024e-03
Train: 49 [ 200/390]  Loss: 0.1761 (0.112)  Acc@1: 95.3125 (96.0665)  Acc@5: 100.0000 (99.9767)LR: 1.024e-03
Train: 49 [ 250/390]  Loss: 0.1585 (0.115)  Acc@1: 93.7500 (95.9661)  Acc@5: 100.0000 (99.9813)LR: 1.024e-03
Train: 49 [ 300/390]  Loss: 0.1063 (0.116)  Acc@1: 96.8750 (96.0185)  Acc@5: 100.0000 (99.9740)LR: 1.024e-03
Train: 49 [ 350/390]  Loss: 0.1075 (0.114)  Acc@1: 95.3125 (96.0871)  Acc@5: 100.0000 (99.9733)LR: 1.024e-03
Train: 49 [ 390/390]  Loss: 0.07028 (0.113)  Acc@1: 97.5000 (96.0840)  Acc@5: 100.0000 (99.9760)LR: 1.024e-03
train_acc 96.084000
Valid: 49 [   0/390]  Loss: 0.4814 (0.481)  Acc@1: 84.3750 (84.3750)  Acc@5: 98.4375 (98.4375)
Valid: 49 [  50/390]  Loss: 0.5195 (0.429)  Acc@1: 84.3750 (85.2941)  Acc@5: 100.0000 (99.5404)
Valid: 49 [ 100/390]  Loss: 0.1764 (0.423)  Acc@1: 93.7500 (85.7519)  Acc@5: 100.0000 (99.5050)
Valid: 49 [ 150/390]  Loss: 0.3311 (0.425)  Acc@1: 90.6250 (85.8133)  Acc@5: 100.0000 (99.4826)
Valid: 49 [ 200/390]  Loss: 0.5532 (0.436)  Acc@1: 82.8125 (85.5100)  Acc@5: 100.0000 (99.4403)
Valid: 49 [ 250/390]  Loss: 0.3083 (0.435)  Acc@1: 92.1875 (85.5889)  Acc@5: 100.0000 (99.3899)
Valid: 49 [ 300/390]  Loss: 0.4619 (0.430)  Acc@1: 84.3750 (85.7091)  Acc@5: 100.0000 (99.4082)
Valid: 49 [ 350/390]  Loss: 0.3440 (0.431)  Acc@1: 92.1875 (85.6704)  Acc@5: 98.4375 (99.4035)
Valid: 49 [ 390/390]  Loss: 0.4160 (0.432)  Acc@1: 85.0000 (85.6880)  Acc@5: 100.0000 (99.4040)
valid_acc 85.688000
epoch = 49   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3420, 0.6580],
        [0.3530, 0.6470],
        [0.3244, 0.6756],
        [0.2979, 0.7021],
        [0.3924, 0.6076],
        [0.4510, 0.5490],
        [0.3738, 0.6262],
        [0.5480, 0.4520],
        [0.5132, 0.4868],
        [0.4035, 0.5965],
        [0.4547, 0.5453],
        [0.4981, 0.5019],
        [0.5091, 0.4909],
        [0.4917, 0.5083]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.2290, 0.7710],
        [0.2063, 0.7937],
        [0.2474, 0.7526],
        [0.2347, 0.7653],
        [0.3197, 0.6803],
        [0.1639, 0.8361],
        [0.2049, 0.7951],
        [0.3223, 0.6777],
        [0.3169, 0.6831],
        [0.2035, 0.7965],
        [0.2808, 0.7192],
        [0.3420, 0.6580],
        [0.3570, 0.6430],
        [0.4204, 0.5796]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
